Arrow in Flight - New Developments in Data Connectivity

on YouTube ↗33m

" summary

Apache Arrow standardizes a columnar memory layout and IPC so projects avoid costly conversions and share data zero‑copy across languages. Real-world wins include Spark UDFs speeding Python execution up to 200x and analytics pipelines claiming up to 100x acceleration; Python bindings hit ~70 million monthly downloads. New infrastructure includes Arrow Flight SQL for columnar client‑server queries, the ADBC driver layer, Arrow Dataset for multi‑file IO, GPU use via RAPIDS, and production tooling like Lance and DuckDB integrations.

AI-written summary · about

" chapters (6)

1. 0:00Apache Arrow intro
2. 5:02Arrow history & success
3. 13:27Arrow ecosystem & integrations
4. 21:33Arrow use cases
5. 31:21Arrow future role
6. 32:46Q&A

" related talks

• 35mPractical Applications of Apache ArrowMatthew Topol, Will Ayd · PyData Virginia 2025
• 42mState of the Apache Arrow Ecosystem:How your project can leverage Arrow!Matthew Topol · Øredev 2023
• 01h 02mApache Arrow: High-Performance Columnar Data FrameworkWes McKinney · Vaccination Database Talks (Second Dose) 2021
• 01h 04mWhere We’re Going, We Don’t Need Rows: Columnar Data Connectivity with Apache Arrow ADBCIan Cook · Future Data Systems Seminar Series (Fall 2025)
• 32mThe expanding Apache Arrow universeJoris Van den Bossche · PyData Paris 2024
• 45mApache Arrow: In Pursuit of SpeedIgor Seliverstov · JPoint 2022
• 25mSpeed Up Data Access with PyArrow Apache Arrow Data is the new APIDeepak K Gupta · PyCon US 2022
• 27mApache Arrow - The columnar format! What's that?Raúl Cumplido · PyCon US 2024