Monitoring Cloud Foundry: Learning about the Firehose
summary
Cloud Foundry’s Loggregator sends binary Dropsonde (protocol buffers) envelopes through a WebSocket traffic controller called the Loggregator firehose; collectors are Metron agents and Dopplers. Consuming apps use “nozzles” (CF plugin or app) — the syslog nozzle injects app names so historical lookups work. Demo shows ~10 MB/min on bosh‑lite (19 VMs), container metrics every ~30s, and a memory-based autoscaler that watches average memory, triggers at 100 MB, and successfully scales a dummy app from 1 to 2 instances. Source code and CF API hooks are published on ECS Team’s GitHub.
AI-written summary · about
chapters (21)
- 0:00Intro
- 1:07Cloud Foundry
- 1:28CICD
- 1:45gridlock
- 2:07jungle
- 2:45logger
- 4:35pill
- 4:52protocol buffers
- 5:40syslog events
- 7:31Firehose
- 9:34Nozzles
- 9:47Firehose User
- 11:42Dropping Ani Format
- 12:08Event Types
- 12:32Lifecycle
- 13:18Demo
- 13:59Log Messages
- 15:21App Log Messages
- 16:53Feedback Control Loops
- 23:52Where to get the code
- 24:04Questions
related talks
29mCloud Foundry Logging & Metrics
32mTuning and Managing Loggregator, the Cloud Foundry Logging Subsystem
31mThe Long and Winding Road to Application Logging [A]
33mSeeing the Forest for the Trees: An Operator’s Guide to Observability of CF with...
10mLightning Talk: Fluentd as syslog drain for CloudFoundry
13mLightning Talk: Cloud Foundry Monitoring: How to Never Leave Your Deployment Unattended
05mCloud Foundry Monitoring & Alerting
25mMonitoring Cloud Foundry Infrastructure and Applications with Stackdriver and BigQuery