Mistakes Were Made
summary
A payments-related feature is extracted from a Rails monolith into a Kotlin + Akka microservice, and the team admits four concrete failures: Kotlin and Akka are chosen with almost no company experience, work is siloed so knowledge leaves when a teammate moves, caching and feature work are prematurely optimized (debugging pain, built ~10,000x overcapacity), and input plus algorithm change happens simultaneously. Despite that the service supports millions of users it eventually yields about $1.5M annual savings.
AI-written summary · about
chapters (13)
- 0:00Intro
- 0:44HELLO!
- 1:33Goal
- 3:13Project Overview
- 3:53REALTIME EXPIRATION
- 5:00The Story Begins.
- 5:44Planning Meeting
- 8:47SILOING WORK
- 10:41PREMATURE OPTIMIZATION
- 14:17TOO MANY CHANGES
- 15:23Verify bite-sized changes
- 16:15BIGO
- 17:02Act before the POST MORTEM
related talks
33mDon’t Get Blamed for Your Choices
32mBuilding for Gracious Failure
47mI’ve Made a Huge Mistake: Implementing Agile on Infrastructure Teams
34mNightmare Before Best Practices
30mLearning from Learnings: Anatomy of Three Incidents
29mEmbracing Failures to Avoid Meltdown
30mHow Not to Build a Service
28mThe Failing CTO - What we can learn from others' mistakes