1 year ago
#155083
vrajkuma
Spring Batch very slow when using 2 datasources - one for Spring Batch and another for the App
I modified this sample batch job provided by spring to use two custom datasources instead of the one autoconfigured by boot. Both datasources point to the same MySql DB server, but to different schemas. One schema for Batch/Task tables and another for the app tables. MySql is running locally. The performance was much much slower compared to the same job running with default boot configured datasource or with ONE custom datasource.
Here is the timing I got and can't figure out why #3 is taking a long time:
- Default Boot configured datasource - 1 second
- One custom datasource (for both Batch/Task and App) - 1 second
- Two custom datasources (one each for Batch/Task and App) - 90 seconds !!!
Do I need to set any CP settings for the custom datasources when using two of them? I tried a few, but didn't help.
Here is the properties file:
spring.application.name=fileIngest
spring.datasource.url=jdbc:mysql://localhost:3306/test-scdf?useSSL=false
spring.datasource.username=<user>
spring.datasource.password=<pwd>
spring.datasource.driverClassName=org.mariadb.jdbc.Driver
app.datasource.url=jdbc:mysql://localhost:3306/test?useSSL=false
app.datasource.username=<user>
app.datasource.password=<pwd>
app.datasource.driverClassName=org.mariadb.jdbc.Driver
Here are relevant portions of my datasource config as recommended here.
@Bean(name = "springDataSource") // for Batch/Task tables
public DataSource dataSource(@Qualifier("springDataSourceProperties")DataSourceProperties springDataSourceProperties) {
return DataSourceBuilder.create().driverClassName(springDataSourceProperties.getDriverClassName()).
url(springDataSourceProperties.getUrl()).
password(springDataSourceProperties.getPassword()).
username(springDataSourceProperties.getUsername()).
build();
}
@Bean(name = "appDataSource") // for App tables
@Primary
public DataSource appDataSource(@Qualifier("appDataSourceProperties") DataSourceProperties appDataSourceProperties) {
DataSource ds = DataSourceBuilder.create().driverClassName(appDataSourceProperties.getDriverClassName()).
url(appDataSourceProperties.getUrl()).
password(appDataSourceProperties.getPassword()).
username(appDataSourceProperties.getUsername()).
build();
I just inject the appropriate datasource into the BatchConfiguration as needed.
@Configuration
@EnableBatchProcessing
public class BatchConfiguration extends DefaultBatchConfigurer {
...
@Override
@Autowired
public void setDataSource(@Qualifier("springDataSource") DataSource batchDataSource) {
super.setDataSource(batchDataSource);
}
@Bean
public BatchDataSourceInitializer batchDataSourceInitializer(@Qualifier("springDataSource") DataSource batchDataSource,
ResourceLoader resourceLoader) {
BatchProperties batchProperties = new BatchProperties();
batchProperties.setInitializeSchema(DataSourceInitializationMode.ALWAYS);
return new BatchDataSourceInitializer(batchDataSource, resourceLoader, batchProperties);
}
spring-data
spring-batch
spring-jdbc
spring-cloud-dataflow
spring-cloud-task
0 Answers
Your Answer