The expanding Apache Arrow universe
summary
Overview of concrete wins and tools: Arrow provides ADBC drivers (BigQuery among them) with a fetch_arrow_table API that avoids rowwise overhead; a nanoarrow C runtime gives tiny, embeddable readers; a new Arrow PyCapsule protocol layers on the C data interface to enable automatic zero‑copy handoffs between data frame libraries; string storage gains a fixed‑width views buffer to cut copies during sorts/filters; a device extension surfaces GPU memory sharing for real deployments.
AI-written summary · about
chapters (10)
- 0:00Apache Arrow intro
- 3:36Columnar memory format
- 8:14Arrow specs and impls
- 11:30New string storage
- 17:15Arrow extension types
- 18:43C Data Interface
- 21:10PyCapsule interop
- 23:23GPU data sharing
- 23:44ADBC for databases
- 26:21Q&A
related talks
42mState of the Apache Arrow Ecosystem:How your project can leverage Arrow!
33mArrow in Flight - New Developments in Data Connectivity
35mPractical Applications of Apache Arrow
01h 04mWhere We’re Going, We Don’t Need Rows: Columnar Data Connectivity with Apache Arrow ADBC
51mUrsa Labs and Apache Arrow in 2019
27mApache Arrow - The columnar format! What's that?
34mLeveraging Apache Arrow for ML Workflows
32mConnecting PyData to other Big Data Landscapes using Arrow and Parquet