Scraping Your Cassandra Metrics - the Right Way, and the Dirty Way
summary
Cassandra metrics generate hundreds of thousands of MBeans and scraping them causes big CPU spikes, so AxonOps builds a Java agent (Byte Buddy) that pre-creates MBean objects and even bypasses the JMX permission check to avoid per-scrape object creation and expensive JDK permission calls. The agent scrapes every 5s, handles 1,000 tables with brief ~33% CPU spikes whereas jmx_exporter fails earlier, completes 1M read/writes about twice as fast, and sends ~700 MB/day vs ~186 GB/day for jmx_exporter (mcac ~5 GB/day).
AI-written summary · about
chapters (7)
- 0:00Introduction
- 3:54Why Cassandra metrics
- 7:29Standard metrics problems
- 11:46Optimizing with Java agent
- 15:04Benchmarking solution
- 21:24Architectural benefits
- 23:51Q&A
related talks
20mMoving Apache Cassandra command execution and monitoring from JMX to CQL: A Developer's Overview
20mAggregating Metrics In-Flight: Challenges, Opportunities
45mJava Performance Analysis and Optimization
30mDay 2 Operations Best Practices with Apache Mesos
33mObservability-First Kafka: Engineering Visibility at Scale
32mScalable Monitoring Using Prometheus with Apache Spark
52mIs your JVM app flying blind? Unmask hidden issues with Observability superpowers!
36mKafka Monitoring: What Matters!