30 minutes of Failure
summary
A string of concrete failures and fixes: using mqj explorer that deletes queues without confirmation; clicking a red stop button in pivotal cloud foundry that kills every service; an incorrect synchronous socket appender in logback that turns a Kafka outage into a 10‑second collapse of 60+ servers (platform runs ~400 apps, ~3k req/s, 150 teams); an Angular memory leak from forgotten unsubscribes that crashes low‑RAM browsers; a find -exec delete run from / that wipes OS files because backups are never tested.
AI-written summary · about
chapters (12)
- 0:00Introduction to failures
- 1:08Bitcoin wallet disaster
- 2:12Open mail relay mistake
- 3:16Server room vacuum incident
- 4:02Improper tooling with MQJ Explorer
- 6:28Stopping services by mistake
- 9:43Angular memory leak crash
- 14:20Banking system collapse in 10 seconds
- 21:07Deleting production files by accident
- 25:23Backup restoration failure
- 27:08Lessons learned from failures
- 30:58Q&A on system failures
