Efficient Resource Management
summary
A production pipeline for high‑QPS asynchronous services across ARM64 and x86_64 and multi‑region deployment (US, JP, IE) that collects JSON structured logs and time‑series metrics, ingests via Kafka ingestion with Logstash/Fluentd, stores metrics in Prometheus + Cortex long‑term storage, visualizes with Grafana dashboards and PagerDuty alerts, and uses Jaeger/OpenTelemetry for tracing. Scaling policy is hybrid (regional horizontal + vertical), log rotation and retention are enforced, and the system is deployed and running in production.
AI-written summary · about
chapters (9)
- 0:00Introduction to efficient resource management
- 5:12Monitoring and logging systems
- 10:05Scaling strategies for services
- 15:40Log management and analysis
- 22:09Prometheus and Grafana setup
- 27:28Prometheus namespaces and alerts
- 31:26Centralized alerting and dashboards
- 32:32Distributed tracing overview
- 33:36Q&A and closing remarks
related talks
30mRunning Highly Scalable Tracing, Logging, Monitoring & Metric System in Production
38mTempering our Production Practices
30mBuilding a Large-Scale, Streaming-Based Logging and Monitoring Solution
27mObservability with Prometheus and beyond
29m“If you Don’t Monitor your Infrastructure, you Don’t Own it!” Regain Control Thanks to Prometheus
33mGive yourself time to sleep
46mBuilding a scalable logging platform
41mResilient Observability at the Retail Edge: A Lightweight, Scalable, and...