State of the Apache Arrow Ecosystem:How your project can leverage Arrow!
summary
Apache Arrow is an in‑memory columnar spec that claims zero‑copy data exchange and shows it with concrete tooling: Arrow Flight / Flight SQL for wire RPC, ADBC client API, Nano Arrow used to shrink the Snowflake Python connector so it fits in AWS Lambda, and DataFusion/Ballista as a Rust compute engine. The project reports ~12.5k GitHub stars and 900+ contributors, PyArrow downloads spike in the tens of millions monthly, and DuckDB/ADBC benchmarks demonstrate large transposition and serialization costs eliminated.
AI-written summary · about
chapters (13)
- 0:00Apache Arrow intro
- 1:29What is Arrow
- 8:56Arrow ecosystem
- 12:35Use Arrow in project
- 17:29Arrow with Spark
- 19:36Arrow Flight RPC
- 21:44Flight SQL & ADBC
- 25:34Arrow Database Connectivity
- 28:45Benchmarking with DuckDB
- 30:10Arrow evolution
- 32:14Call to action
- 34:46Resources
- 35:55Q&A
related talks
33mArrow in Flight - New Developments in Data Connectivity
35mPractical Applications of Apache Arrow
32mThe expanding Apache Arrow universe
09mBackstage interview
51mUrsa Labs and Apache Arrow in 2019
34mApache Arrow as a full stack data engineering solution
01h 04mWhere We’re Going, We Don’t Need Rows: Columnar Data Connectivity with Apache Arrow ADBC
27mApache Arrow - The columnar format! What's that?