Architecting Prometheus for cost-efficiency

on YouTube ↗04m

" summary

Claim of 5–10x cost savings by running Prometheus-style ingestion on serverless compute with a S3-only storage backend, avoiding SSDs and global indexes; a custom in-memory representation and caching cut compute 10x and query latency stays under 3s. Architecture pins sender and receiver to AZs to achieve zero cross-AZ egress and reports 14B datapoints/min, 15M series/hour, ~3TB/day on the live demo.

AI-written summary · about

" related talks

• 49mUsing serverless functions for real-time observabilityLiz Fong Jones · CPH DevFest 2024
• 25mMonitoring in a Serverless WorldColin Douch · OSMC 2022
• 32mBuilding and scaling a log analytics platform - a serverless approachNarendran · Rootconf Cloud Server Management Miniconf Chennai 2017
• 30mOperating Prometheus in a Serverless WorldColin Douch · KubeCon + CloudNativeCon Europe 2022
• 51mArchitecting for Hyperscale with AWS: Navigating Real-world Serverless ChallengesPawan Puthran · Great International Developer Summit 2024
• 29mHow to build a social network on serverlessYan Cui · apidays Barcelona 2019
• 47mAdvanced Serverless Architectural Patterns on AWSAlex Casalboni · Heapcon 2019
• 41mUsing Serverless & ARM64 for Real-Time ObservabilityLiz Fong Jones · GOTO Chicago 2024