Scaling Batch Applications
summary
Shows how Spring Batch 5 jobs actually scale with concrete numbers and code: uses async item processor + async item writer to cut a 10,000‑row job from 45s (processor sleep) down to ~10s, uses multi‑threaded task executor to parallelize chunks, and uses remote chunking via RabbitMQ (Spring Integration) to distribute chunks across worker JVMs (example 10k completed in ~11s). Notes need for the JobRepository tables, the enableBatchProcessing change in SB5, and chunk‑size tradeoffs (100k simple ≈7m vs async ≈1m40s).
AI-written summary · about
chapters (20)
- 0:00Introduction to batch jobs
- 3:23Spring Batch fundamentals
- 9:10Step types and processors
- 14:48Five scaling techniques
- 15:55Parallel steps and flows
- 17:22Async processor and writer
- 19:11Multi-threading in Spring Batch
- 20:33Remote partitioning and chunking
- 25:59Demo setup and job definition
- 28:32Simple job performance test
- 29:36Impact of processor delays
- 31:20Async processor performance
- 33:30Multi-threading performance
- 35:16Remote chunking demo
- 40:58Remote chunking Q&A
- 45:13Combining scaling techniques
- 46:01Performance tuning factors
- 50:15Chunk size optimization
- 53:03Benchmark results summary
- 54:49Resources and Q&A
related talks
49mBatch Processing in Action
01h 01mSpring Batch 5.2 and Beyond: What’s New and What’s Coming Next
44mBatching for the Modern Enterprise
01h 24mJava Batch JSR-352
01h 07mJBatch or Not-So-Big Data
28mBatch Processing with Spring Batch and Kubernetes
22mWhat’s New in Spring Batch?
47mWhat's new in Spring Batch 5