Troubleshooting In Distributed Systems
summary
Recommendation: design services to fail well and emit expressive logs with stack traces and structured fields; centralize shipping with Logstash forwarder into elk where Logstash parses/transforms (it has 50 output plugins), Elasticsearch indexes for fast REST queries, and Kibana builds dashboards that reveal spikes and correlated events; anonymization, multiline stack parsing, and tag-based filtering are implemented; concrete demo shows dashboards and that the pipeline works end to end for investigation and postmortem evidence.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 0:40Scaling Up
- 3:22Failure
- 5:51Logging
- 10:38Log Stash
- 17:18Cabana Interface
- 19:08Line Chart
- 20:58Filtering
- 23:03Dashboard
related talks
40mA needle in a logstack
01h 00mApplication Logging & Logstash
42mMake your logs work for you
01h 02mAdding Observability to Distributed Systems
49mImproving Observability of Distributed Systems
30mBuilding a Large-Scale, Streaming-Based Logging and Monitoring Solution
43mPractical Application Logging and Monitoring
30mCentralized Log Management