The AppMetrica API from the inside, or SQL without SQL

on YouTube ↗42m

" summary

An inside look at AppMetrica’s backend for mobile analytics, describing its architecture and how it ingests and stores billions of real-time events. Covers storage and processing approaches (different ways to store and handle events), the data model used for reports, example analytics tasks and API queries, how request processing is implemented in code, and pros/limits of their custom query language (SQL-like but not SQL).

AI-written summary · about

" chapters (14)

1. 0:00Introduction
2. 2:11AppMetrica architecture overview
3. 4:57ClickHouse in AppMetrica
4. 7:08Table organization and partitioning
5. 10:25Reporting API format and tasks
6. 13:17Simple Reporting API queries
7. 16:46ClickHouse query execution
8. 18:08Complex queries: table joins
9. 23:01Data locality and optimization
10. 25:10ClickHouse data sampling
11. 28:47Adaptive sampling
12. 30:32Offset and memory handling
13. 32:20Summary and system limits
14. 34:26Q&A

" related talks

• 37mDeduplication of 5 Million Events per Second on YDB in AppMetricaArtem Ismagilov · Saint HighLoad++ 2022
• 25mBuilding an analytics APIDavid Wobrock · apidays Paris 2021
• 31mAction-Packed AnalyticsSteve Ganem · Firebase Dev Summit 2016
• 41mStep By Step Guide to Building an Application Analytics SystemAnton Lavrik · Erlang Factory SF Bay Area 2014
• 39mEmbedded Analytics through Data APIDionizas Antipenkovas · Big Data Conference Europe 2019
• 45mData as culture as softwareAndrea Burbank · Crunch Conference 2019
• 26mMetrics-driven Data ArchitecturesNed Letcher · DataEngBytes 2021
• 12mAnalytics architecture strategy for intelligent experiencesKuber Sharma · AWS re:Invent 2022