Go at Mainframe Scale
summary
Built a distributed Go system that ingests Capital One account data at 2 million tps and peaks around 5 Gbps, persisting over 20 TB across multiple regions in AWS. The project creates internal Go libraries on the job, defines microservice boundaries, and ramps teams with few prior Go engineers while relying on a single Go SME to guide standards. It scales to production, processes hundreds of millions of requests, and even enables shutting down a mainframe.
AI-written summary · about
chapters (27)
- 0:00Intro
- 1:02Go at Mainframe Scale • Part 1: Mainframe Scale
- 2:01The Mainframe Origin Story • Howard Alken created the first Mainframe at IBM, called the Harvard Mark I in 1944
- 4:49Scale Today
- 6:29Who is still using mainframes Communications
- 7:54How we want to mainframe merges to work
- 9:05How they actually work
- 13:29Technical Challenges
- 16:296 months in
- 17:47Failed attempts at replacing mainframes
- 18:33My Mainframe Scale Leamings
- 22:03Mainframe Adjacent Systems
- 23:41Gopher phase
- 28:11Experience across an organization
- 29:33Language Background
- 32:07Story: Grace Hopper
- 36:24MVT Objectives
- 38:06Interfaces
- 42:35Typing
- 43:18Library Development
- 44:24Why manually test? Case Study: Systematics
- 45:08Testing Best Practices
- 46:25The Poor (wo)man's Code Review
- 47:31Banking Systems
- 48:46Upsetting Concurrency Findings Where we need it: • Saving into databases (sometimes) • Making parallel calls to dependencies Where don't we need it! . Most of our systems
- 50:50Total Volume
- 52:03IBM Open Enterprise SDK for Go is now available!
related talks
24m"Go Big! Go Enterprise"
31mHow Go is Making us Faster
06mHow We Achieved 100x Scale
28mManaging and Scaling Real Time Data Pipelines using Go
45mMigrating Mission Critical Workloads from Mainframe to Open...
27mScaling using Go
37mThe Need For Speed: Scaling Go Microservices - Artur Kondas
30mHigh Performance Systems in Go