Migrating Workloads from Mainframes to Cassandra - Problems, Experience and Learnings
summary
U.S. Bank moves read traffic off mainframes to Cassandra using a CQRS pattern, Kafka and Spark (structured/continuous streaming) and writes remain on the mainframe. Pipeline adds a publisher on the mainframe, Kafka with tuned partitions, Spark checkpointing and Cassandra batching; peak ~300 TPS and billions of rows. Claim: 95% of events land under 1 second (most under 500 ms). Missing events use a short-lived journal ID and a nightly/30‑minute data reconciliation job to find and repair gaps.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 3:16Digital transactions growth
- 7:52Mainframe architecture overview
- 9:15Read-heavy workload migration
- 11:24Cassandra migration challenges
- 13:52Latency under one second
- 17:06Preventing data loss
- 19:35Data reconciliation
- 21:20Key takeaways
- 22:27Q&A
related talks
45mMigrating Mission Critical Workloads from Mainframe to Open...
26mBuilding Financial Systems on Eventually Consistent DBs
21mModeling Financial Data In Cassandra To Serve Real Time And Batch Workloads
55mSolving Classical Data Analytic Task by Using Modern Distributed Databases
46mApache Cassandra: 8 years in production
24mMigrations under Production Load: How to Switch...
35mMission Critical Data, and Evaluating the Community in OSS Tech
32mCassandra at Yammer #bbuzz