Making Noise
summary
A production API gateway causes repeated 500s for three hours under modest load; replacing blocking Task.await calls with Task.await → Task.yield stops cascading crashes, moving single-service requests into Fastly VCL bypass reduces gateway load, and a temporary Ruby Kafka consumer fixes unstable Elixir drivers. Response time stabilizes near 100 ms, availability improves and traffic records repeat twice later. Service ownership, biannual reviews and “ghosting” (duplexing) real traffic enable safe, incremental rollouts.
AI-written summary · about
chapters (30)
- 0:00Intro
- 1:39System Failure
- 3:17API Gateway
- 4:13Single Service Requests
- 4:52What Happened
- 9:45Wahby
- 10:45WabiSabi
- 13:47Systems
- 14:36System Life Cycle
- 16:10Secondary Information
- 17:00App Architecture
- 17:52Tasks await
- 19:39Rethinking our architecture
- 21:17Response times
- 22:12Technology isnt static
- 22:54Adding new technology
- 23:21Kafka
- 25:58Maintaining
- 26:17People lifecycle
- 26:52Trusted autonomy
- 27:55Service owners
- 29:23Application reviews
- 30:09Bidirectional ideas
- 32:33No one is irreplaceable
- 34:34Impermanent architecture
- 35:39Non dogmatic thinking
- 36:19Infinite number of solutions
- 36:57Shared responsibilities
- 37:41Growth decay
- 39:08Monitoring
related talks
30mStupid Ideas For Many Computers
31mIn the kitchen: A sprinkle of fire and Chaos
41mOrchestrating the Web Audio API
26mCrafting crazy & collectively constructive creative coding concoctions
29m25 years of embracing the weird
41mWhere the Wild Things Are
23mLightning Talks Pt 2
52mForce Multiply – The art and science of accelerating workflows in CG pipelines