Batch Processing with Spring Batch and Kubernetes

on YouTube ↗28m

" summary

Shows a working remote-partition Spring Batch setup that uses RabbitMQ request/reply queues and a database-backed job repository to distribute file-processing partitions from a master to workers, deployed on a Minikube cluster. Demo runs with 2 worker replicas, each processing distinct files, logs and job tables confirm partitions complete, and scaling replicas down to one shifts all work to the remaining pod; restartability and reprocessing of failed partitions are demonstrated.

AI-written summary · about

" chapters (8)

1. 0:00Introduction to batch processing
2. 2:39What is Spring Batch
3. 5:06Spring Batch architecture
4. 9:06Partition technique explained
5. 15:29Kubernetes for batch scaling
6. 18:14Kubernetes cluster setup
7. 19:30Live demo walkthrough
8. 27:09Q&A and closing

" related talks

• 49mBatch Processing in ActionRodrigo Graciano, Hillmer Chona · Spring I/O 2022
• 01h 00mScaling Batch ApplicationsRodrigo Graciano · Devnexus 2023
• 41mHow Kubernetes and Spring Batch Process Over 12,000 Hours of Video per MonthEuber Bispo · QCon São Paulo 2019
• 40mDeploying Kafka Streams Applications...Gwen Shapira, Matthias Sax · Scale By The Bay 2018
• 40mScaling microservices with Message queues, .NET and KubernetesLewis Denham Parry · NDC London 2019
• 51mJava-based microservices, containers, Kubernetes - how toRay Tsang · J-Fall 2015
• 24mKubernetes Batch Processing at Scale - A Scheduling PerspectiveDeliang Fan, Lim Haw Jia · Kubernetes Batch + HPC Day Europe 2023
• 46mAsynchronous Job Processing in Java with JobRunrKo Turk · JCON EUROPE 2025