What and Why and How: Apache Drill 1.0 #bbuzz
summary
Apache Drill 1.0 claims to run SQL directly over JSON and files without ETL by using late-binding SQL with any type that observes an empirical schema per row batch, regenerates and compiles code (Janino/bytecode) to remove conditionals, and stores data in ValueVectors and vectorized in-memory execution for high throughput. It demonstrates flatten and KV_GEN for nested arrays/objects, parquet pushdown readers, quick untar install or a five‑node demo cluster, and a view-based security mapped to filesystem permissions with tests up to ~1,000 nodes.
AI-written summary · about
chapters (11)
- 0:00Intro
- 3:03Punk SQL: SQL for a Hadoop World Modern Syntax
- 6:53Drill Provides A Flexible Data Model
- 8:21Flexibility in how you describe your data • Drill doesn't require schema, detects file types based on - extensions
- 11:49Intuitive SQL access to complex data
- 14:30Extensions to ANSI SQL to work with repeated values
- 21:12Secure your data without an extra service
- 27:13Parquet: The Format for the next decade Apache Drill & Apache Parquet communities working together • Better Ecosystem integration and decentralized metadata
- 29:25Runtime Compilation Pattern
- 31:29A Color-coded visual layout and Gant timing chart is provided
- 33:45Scale and Concurrency
related talks
31mVersatility and Functionality of ApacheDrill
44mNonrelational SQL
36mTaming JSON with SQL: From Raw to Results
32mSome Like It Hot
31mApache Gravitino: A Multi-regional, Geo-distributed Meta Datalake
01h 03mThe Future of Apache Hive and Hadoop 2.0: Bigger, Faster, Stronger
01h 04mQuery Optimization and Acceleration at Dremio
27mTales at Scale: Analytics at 1000 QPS and Beyond