Apache Kudu (incubating): Fast Analytics on Fast Data #bbuzz
summary
Apache Kudu is a C++ tabular storage engine that uses Raft-replicated tablets (default three replicas) and removes the HDFS layer so clients talk directly to tablet servers; a single-node RAM benchmark hits 750,000 random reads/s, typical clusters aim for millions of IOPS and multiple GB/s sequential per node, and a 75-node TPC‑HB run shows Kudu+Impala 31% faster than Parquet on a 100 GB in‑RAM dataset while Kudu is 40–50% slower when data outgrows RAM. Integrations include Impala, Spark SQL and a Java/C++/Python API; Xiaomi runs ~20 billion rows/day with 10 ms–10 s ingest latency; schema changes are constant-time ALTER TABLE.
AI-written summary · about
chapters (10)
- 0:00Introduction and motivation
- 1:30Why build Kudu
- 8:49What is Kudu
- 12:01Use cases and architectures
- 17:41Real-world example Xiaomi
- 21:35How Kudu works distributed
- 26:14Storage design for analytics
- 33:15Performance benchmarks
- 36:30How to try Kudu
- 36:53Q&A
related talks
45mUsing Apache Kudu for Analytical Workloads
33mKudu
39mAccelerate big data analytics with Apache Kylin
31mBuilding large scale, transactional data lakes with Apache Hudi
28mThe Heat is On: Architecting for Hot Analytics
28mAnalyze data easily with duckdb - and the implications on data architectures
32mSome Like It Hot
29mRealtime Analytics Unleashed: Unlocking the Potential of Data at Scale