Batch Processing in Spring (Spring Batch)

Batch processing means processing large volumes of data in chunks (small groups) instead of one by one. It’s useful when you need to:

• Import/export thousands of records.

• Generate monthly reports.

• Process payroll for employees.

• Clean up old data.

Spring Batch is a framework in Spring that makes batch processing easier by handling:
✔ Reading data (from DB, CSV, etc.)
✔ Processing data (modifying/validating)
✔ Writing data (to DB, file, etc.)
✔ Retrying failed jobs
✔ Scheduling jobs (run daily, weekly, etc.)

Key Components of Spring Batch

1. Job

• A batch process (e.g., "Generate Monthly Sales Report").

• Contains one or more steps.

2. Step

• A single unit of work in a job.

• Has:

  • Reader → Reads data (e.g., from a CSV file).

  • Processor → Processes data (e.g., calculates total sales).

  • Writer → Saves results (e.g., writes to a database).

3. JobLauncher

• Starts the Job when triggered (manually or via scheduler).

4. JobRepository

• Stores metadata (which jobs ran, failed, succeeded).

Example: Processing Employee Bonuses (Step-by-Step)

Scenario

• You have a CSV file (employees.csv) with employee data:

id,name,salary 1,John,5000 2,Alice,7000 3,Bob,6000

• You need to give a 10% bonus to each employee and save it to a database.

1. Define the Job (Batch Process)

@Configuration @EnableBatchProcessing public class BonusJobConfig {

@Autowired private JobBuilderFactory jobBuilderFactory;

@Autowired private StepBuilderFactory stepBuilderFactory;

@Autowired private DataSource dataSource; // Database connection

// Step 1: Read CSV → Process Bonus → Write to DB @Bean public Step bonusCalculationStep() {
return stepBuilderFactory.get("bonusCalculationStep")
.<Employee, Employee>chunk(10) // Process 10 records at a time
.reader(employeeReader())
.processor(employeeProcessor())
.writer(employeeWriter()) .build();
}

// Define the Job @Bean public Job calculateBonusJob() {
return jobBuilderFactory.get("calculateBonusJob")
.start(bonusCalculationStep())
.build();
}
}

2. Define Reader (Reads CSV File)

@Bean public FlatFileItemReader employeeReader() {

return new FlatFileItemReaderBuilder() .name("employeeReader")
.resource(new ClassPathResource("employees.csv")) // Reads CSV
.delimited() .names("id", "name", "salary") // Maps CSV columns
.targetType(Employee.class) // Converts to Employee object
.build();
}

3. Define Processor (Calculates 10% Bonus)

public class EmployeeProcessor implements ItemProcessor<Employee, Employee> {
@Override public Employee process(Employee employee)
{
double bonus = employee.getSalary() * 0.10;
employee.setBonus(bonus); return employee;
}
}

@Bean public EmployeeProcessor employeeProcessor() {

return new EmployeeProcessor();
}

4. Define Writer (Saves to Database)
@Bean public JdbcBatchItemWriter employeeWriter()
{
return new JdbcBatchItemWriterBuilder()
.dataSource(dataSource)
.sql("INSERT INTO employees_bonus (id, name, salary, bonus) VALUES (:id, :name, :salary, :bonus)") .beanMapped() // Maps Employee fields to SQL query .build();
}

5. Run the Job
You can trigger it manually:
@Autowired private JobLauncher jobLauncher;

@Autowired private Job calculateBonusJob;

public void runJob() throws Exception {
JobParameters params = new JobParametersBuilder()
.addLong("time", System.currentTimeMillis())
.toJobParameters();

jobLauncher.run(calculateBonusJob, params); // Starts the job
}
Or schedule it (run every night at 12 AM):
@Scheduled(cron = "0 0 0 ?") // Runs at midnight daily public void runJobNightly() throws Exception {
runJob();
}

What Happens When the Job Runs?

1. Reader → Reads employees.csv (10 records at a time).

2. Processor → Adds a 10% bonus to each employee.

3. Writer → Saves the updated data to the database.

4. JobRepository → Keeps track of job success/failure.

Why Use Spring Batch?

✅ Handles large data efficiently (processes in chunks).
✅ Recovers from failures (can retry failed steps).
✅ Supports scheduling (run jobs automatically).
✅ Tracks job history (know which jobs ran successfully).

Real-World Use Cases

• Banking: Daily transaction reports.

• E-commerce: Monthly sales analytics.

• HR Systems: Payroll processing.

• Data Migration: Moving data from old to new systems.

Final Summary