Chaos: breaking your systems to make them unbreakable

on YouTube ↗20m

" summary

Company runs 30-minute mini game days with three roles: game master, responder, scribe to cut prep from 90–120 minutes to a half hour. Workflow uses Donut to form three-person Slack groups, Google Calendar/Zoom, a Google Form runbook for live notes, Gremlin for shutdown/blackhole/CPU attacks, and Jira to create one prioritized ticket per experiment. Scenarios test concrete hypotheses (kill a pod, raise CPU, block a dependency), validate monitoring/alerts, and verify fixes with a follow-up run.

AI-written summary · about

" chapters (6)

1. 0:00Intro
2. 5:00Chaos Engineering
3. 7:48Challenges
4. 10:45Mini Game Days
5. 16:07Recap
6. 17:39Resources

" related talks

• 47mChaos Engineering Bottom UpBart Szulc · Build Stuff 2020
• 39mChaos: Breaking your Systems to Make them UnbreakableJason Yee · TestCon Europe 2019
• 46mEmbracing Collaborative Chaos: Chaos Days on Large PlatformsLyndsay Prewer · Agile India 2020
• 37mChaos Engineering: The Fine Art of Breaking Stuff in Production on PurposeGeert van der Cruijsen · DevDays Europe 2019
• 45mGameDays: Practice Thoughtful Chaos EngineeringHo Ming Li · GOTO Berlin 2018
• 48mChaos Engineering: The art of breaking stuffGeert van der Cruijsen · Future Tech 2019
• 15mIncident Ready: Chaos Engineer Your Incident ResponseRobert Ross · Conf42 Chaos Engineering 2021
• 26mBreaking Systems for ResilienceDhiraj Kumar Jain · Conf42 Chaos Engineering 2024