Building Observability Pipeline with Fluent-Bit
summary
LinkedIn replaces siloed logging and tracing stacks with a unified pipeline that uses OpenTelemetry for multi‑language instrumentation, Kafka as the central storage/transport, and Azure Data Explorer for queryable logs. Fluent‑Bit is extended to act as the transport agent with OTLP/protobuf over HTTP, built‑in metrics emission and richer tag management, and deployed in a per‑instance sidecar hybrid model versus per‑host. Work includes sampling, data partitioning, and rollouts to many clusters; Java‑only limits are removed and resource efficiency is cited as a win.
AI-written summary · about
chapters (5)
related talks
33mFluent Bit V2.0: Unifying Open Standards For Logs, Metrics & Traces
36mFluent Bit: Logs, OpenMetrics, and OpenTelemetry all-in-one
22mLogging: Fluentd & Fluent Bit
18mHow To Supercharge AI/ML Observability With OpenTelemetry and Fluent Bit
36mDeep Dive: Fluent Bit
27mBuild a Cloud Native Logging Pipeline on the Edge with Fluentbit Operator
23mCustomized Cloud Native Logging: Fluent Bit in Open Service Mesh
22mFluent Bit - Swiss Army Tool of Observability Data Ingestion