Decentralizing SaaS Monitoring Using Open-Source Tools and Standards
summary
84codes rebuilds observability for ~15,000 VMs running RabbitMQ and LavinMQ by deploying a custom OpenTelemetry Collector distribution per server, using Prometheus receivers, AMQP and prometheus_remote_write exporters, and GitHub Actions to build and validate configs generated in Ruby. Long-term metrics go to Grafana Mimir long-term storage, logs to Loki for logs, rollout hits ~10,000 servers in two days, reduces cross-region latency, and keeps collector CPU/memory under ~1%.
AI-written summary · about
related talks
25mOpen source observability for private cloud: mission impossible or not?
28mImprove Monitoring and Observability for Kubernetes with OSS Tools
23mImplementation of Unified Observability at Scale From Scratch
30mIs Building an Observability Platform with Open Source the Right Choice?
29m“If you Don’t Monitor your Infrastructure, you Don’t Own it!” Regain Control Thanks to Prometheus
56mFull-stack observability and application monitoring with
33mGive yourself time to sleep
28mMonitoring Rails apps using Open Source