Using OTP / Presence for Real-time Analytics
summary
A realtime dashboard polls the analytics DB every five seconds and spins a per-organization supervision tree that launches four GenServer workers per subscription to fetch geo, time‑series and other aggregates; subscriptions are registered globally across nodes and protected with process monitors to recreate workers on node failure. Presence is used server‑side to detect empty topics and stop trees, the whole system is under 500 lines, runs in production for ~100 customers and is feature‑gated.
AI-written summary · about
chapters (9)
- 0:00Introduction and background
- 3:59Real-time analytics requirements
- 7:13Designing the data pipeline
- 10:26Handling duplicate processes
- 13:14Global process registration
- 15:03Managing process removal
- 18:35System considerations and trade-offs
- 22:37Live dashboard demonstration
- 23:00Q&A: Database and backpressure
related talks
20mUsing OTP in Your Phoenix App
47mOTP
39mReal-Time Event Processing and Handling Stateful Applications on Apache Mesos
29mRealtime Analytics Unleashed: Unlocking the Potential of Data at Scale
18mReal-Time Monitoring of Distributed Systems
38mRealtime Data Activation - Analytics, Intelligence & Decision Making
49mIntroduction to Real-Time Analytics with Apache Pinot
47mBuilding a Real-Time Analytics Database