Can Apache Pinot replace your OLTP database?

on YouTube ↗37m

" summary

Compares Apache Pinot to OLTP using a live Docker demo that wires MySQL → Debezium → Kafka → Pinot and shows CDC-driven real‑time upserts and backfill with the Minion bootstrap; demonstrates joins and per‑state aggregations with sub‑second latency. Explains columnar segments, bloom pruning, inverted/range/geospatial indexes, star‑tree index, HLL/Theta sketches and compaction. Claims production scale: LinkedIn clusters at ~200k QPS, billions of keys, petabyte scale; conclusion: Pinot can replace OLTP for hybrid serving and user‑facing analytics but not all transactional workloads.

AI-written summary · about

" chapters (12)

1. 0:00Introduction and agenda
2. 1:06What is OLTP database
3. 3:13OLTP use cases and challenges
4. 10:46Apache Pinot overview
5. 13:14Offloading analytics to Pinot
6. 17:43Pinot data bootstrap and backfill
7. 19:11Replacing OLTP with Pinot
8. 21:00Pinot query optimization techniques
9. 27:24Pinot indexes and performance
10. 29:07Pinot real-time upserts
11. 32:35OLTP vs Pinot comparison
12. 35:03Conclusion and Q&A

" related talks

• 54mAn Introduction to Apache PinotTim Berglund · Devnexus 2022
• 34mNext generation OLAP stack using Apache PinotChinmay Soman · Berlin Buzzwords 2022
• 49mIntroduction to Real-Time Analytics with Apache PinotTim Berglund · GOTO Chicago 2023
• 44mUSER-FACING, REALTIME ANALYTICS USING APACHE PINOTNeha Pawar · Crunch Conference 2021
• 47mReal-time analytics for your users with Apache PinotF. Teychene · DevOps D-Day 2022
• 39mFaster Analytics for Fast Data with Apache Pinot and Flink SQLChinmay Soman · Berlin Buzzwords 2021
• 38mReal-Time Analytics: Going Beyond Stream Processing with Apache PinotKarin Wolok, Rong Rong · Open AI + Data Forum @ Open Source Summit North America 2022
• 34mReal-Time Analytics: Going Beyond Stream Processing with Apache PinotMark Needham, Karin Wolok · Open Source Summit Latin America 2022