Building an analytics API
summary
A proprietary analytics API surfaces a JSON DSL called BQL JSON DSL that lets clients declare metrics, dimensions, filters and time windows; a planner chooses between backends (BigQuery, Snowflake, Athena, key‑value stores, S3) and composes transformers that run SQL, fetch compressed HTML from S3, then orchestrate exports (HTTP, file, or streaming into BigQuery). Lessons: onboarding is hard, fine‑grained monitoring of query cost and latency is essential, and an open alternative to try is Cube.
AI-written summary · about
chapters (12)
- 0:00Introduction to analytics APIs
- 1:27Analytics vs relational data
- 3:53REST APIs for relational vs analytical
- 5:39Butterfly use case
- 8:28Introducing BQL
- 9:32Analytics API architecture
- 13:00Performance and cost optimization
- 15:27Lessons learned
- 17:37Monitoring challenges
- 19:02Cube.js alternative to BQL
- 20:25Key takeaways
- 21:11Q&A
related talks
45mBuilding an Analytics Platform
43mAnalytics capabilities architectural patterns: A journey for data-driven insights
50mMasterclass: Building a data stack in 2021
12mAnalytics architecture strategy for intelligent experiences
33mAnalytics Platform at Petabyte Scale
28mHow to help BI & AI people as a software engineer?
39mBREAKING YOUR DATA TEAM OUT OF THE SERVICE TRAP
30mRaw Data to Business Insights: What You Need to Build a Modern Data Lake