Schrödinger's Pokémon: Observability for Chaotic Load Testing
summary
Demonstrates a reproducible setup: a PokeAPI app on Kubernetes (DigitalOcean) with Prometheus + Grafana for metrics and k6 with xk6-chaos to run load and kill pods from the same script. Baseline 10 users yields 95th percentile 4.223s and ~1.16% errors; a 40-user run spikes the 95th to 17.66s while errors stay ~1.2%. Built chaos functions that terminate app/web pods; observability finds CPU throttling from resource limits and explains performance regressions, allowing targeted fixes.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:05What is chaos engineering
- 3:33Four steps of chaos engineering
- 5:15Chaos engineering as testing
- 7:21Overcoming tool limitations
- 8:27Steady state for Pokémon API
- 10:12Observability with Prometheus and Grafana
- 13:45Baseline with k6 load test
- 18:43Chaos testing hypotheses
- 20:28Running chaos experiments
- 22:19Extending k6 with xk6 chaos
- 24:31Advantages of chaos testing
- 27:44Live demo: terminating pods
- 30:32Analyzing chaos results
- 32:16Discovering CPU throttling
- 36:29Monitoring vs observability
- 37:32Why observability matters
- 38:56Q&A: Implementation
- 41:23Q&A: When to start
- 43:51Closing remarks
related talks
36mConfidence with Chaos for Your Kubernetes Observability
30mBring Your Chaos to Work Day
37mConfidence with Chaos for your Kubernetes Observability
35mChaos Engineering WG Deep Dive
28mChaos Validation Made Easy
43mEmergent Load Testing Rules for Organized Chaos
24mFreedom of Kubernetes requires Chaos Engineering to shine in production
29mCase Study: Bringing Chaos Engineering to the Cloud Native Develo...