Cloud-Native Batch Processing with Spring Batch 4 - Michael Minella

on YouTube ↗01h 10m

" summary

A working pipeline pulls multiple files from S3, maps delimited rows to a Foo object, calls a REST enrichment service, and writes batched rows to MySQL using Spring Batch 4 with builder-based readers/writers. A circuit breaker (Spring Retry) limits failing REST calls to just three hits while the database records continue, avoiding costly rollbacks. Dynamic parallelism uses Spring Cloud Task to launch worker JVMs (Cloud Foundry or local) per partition, and orchestration via Spring Cloud Data Flow successfully runs and monitors the jobs.

AI-written summary · about

" chapters (10)

1. 0:00Intro
2. 1:39SPRING CLOUD TASK
3. 3:11SPRING BATCH 4.0
4. 3:40SPRING BOOT
5. 5:11SPRING FRAMEWORK 5
6. 21:04SPRING CLOUD CONFIG SERVER
7. 21:17SPRING CLOUD EUREKA SERVER
8. 28:10SCALING SPRING BATCH
9. 48:38SPRING CLOUD DATA FLOW
10. 59:01CLOUD NATIVE WITH SPRING

" related talks

• 49mBatch Processing in ActionRodrigo Graciano, Hillmer Chona · Spring I/O 2022
• 44mBatching for the Modern EnterpriseMichael Minella · SpringOne 2020
• 50mWalking Through Spring Cloud Data FlowGlenn Renfro, Ilayaperumal Gopinathan · SpringOne 2020
• 44mData Processing With MicroservicesMichael Minella · Spring I/O 2017
• 01h 17mMessage Driven Microservices in the CloudDr. Mark Pollack, Dr. David Syer · SpringOne2GX 2015
• 01h 32mCloud Native Data Flow OrchestrationMark Fisher, Patrick Peralta · SpringOne2GX 2015
• 40mBuilding Production Data Pipelines by using TDD and Pair ProgrammingDavid S. Miller, Cahlen Humphreys · Data Con LA 2021
• 31mBridge to the CloudSteve Howard · Indy Cloud Conf 2020