Centralized Logging Solution for Google Cloud Platform
summary
Practical guide to centralizing logs on GCP: use Stackdriver Logging with the open‑source agent Fluentd (Google Fluentd) deployed as a DaemonSet on GKE, or the lightweight Fluentbit for high‑volume nodes; route logs with sinks to BigQuery, Cloud Storage, Pub/Sub and trigger Cloud Functions; use Error Reporting and Logs‑Based Metrics to alert (example bug: negative‑quantity error auto‑detected and reopens issue); note scale numbers: Fluentd ~700 plugins, Fluentbit ~45 plugins and ~50k installs/day, projected 43 GB/month vs 50 GB free.
AI-written summary · about
chapters (12)
- 0:00Start
- 3:36Managing logs via a central router
- 7:58Logging Agent
- 9:38Logging Tools
- 23:31Logging in Kubernetes
- 25:00Applications in Kubernetes
- 27:49Logging Context in Kubernetes
- 29:32Log Processing in Kubernetes
- 31:57The value of structured logs
- 33:59Data driven decision making for logs
- 36:07Ecosystem
- 36:43Fluent Bit
related talks
25mMonitoring Cloud Foundry Infrastructure and Applications with Stackdriver and BigQuery
46mBuilding a scalable logging platform
29mAnonymize application logs in DevOps way
33mCloud Native Logging 101 [B]
26mKubernetes and Logging: Do It Right
22mPetabyte Scale Logging with Fluentd and Fluent Bit: A Use Case From...
22mLogging: Fluentd & Fluent Bit
27mBuild a Cloud Native Logging Pipeline on the Edge with Fluentbit Operator