Why Do My Logs Suck?! A Practical Guide to Improving Your Logs
summary
Couchbase moves from scattered, multi-language files to a Fluent Bit sidecar that parses and normalizes logs without changing source code, using file-based tags to route audit data to different endpoints and filters to collapse varied log levels into a constrained enum. Regression tests run known-input/expected-output diffs in CI, a watcher restarts Fluent Bit for dynamic config, and a preprocessor handles giant one-line JSON rebalance reports added newline. The solution is open source, deployed in Kubernetes and on-prem, and works in production.
AI-written summary · about
chapters (11)
- 0:00Start
- 1:05What do we need to handle?
- 7:06Iterate and test!
- 9:06Watcher process
- 10:37How did we do it?
- 12:47Good times!
- 14:47Rapid unscheduled disassembly
- 16:47Bodges
- 18:57Look to the future!
- 19:57More details
- 20:42Q&A time!
related talks
40mLive, Laugh, Log
26mKubernetes and Logging: Do It Right
22mLogging: Fluentd & Fluent Bit
39mMaking Logs Work for you with Fluentd
34mFluentd: A Complete Logging Ecosystem for Kubernetes
34mGetting Started With Logging in Kubernetes
28mBuilding Observability Pipeline with Fluent-Bit
46mBuilding a scalable logging platform