Backstage interview

on YouTube ↗09m

" summary

Apache Arrow's concrete claims and ecosystem work matter here: a memory layout that is identical on wire and in memory enabling zero copy communication, concrete adopters like pandas polars DuckDB and Spark Connect, and the ADBC API with a Snowflake ADBC driver already shipping. A new company called Columnard launched in April building ADBC drivers and tooling with three cofounders and one hire and reports 10–20× speedups by avoiding serialization and conversion.

AI-written summary · about

" related talks

• 42mState of the Apache Arrow Ecosystem:How your project can leverage Arrow!Matthew Topol · Øredev 2023
• 34mLeveraging Apache Arrow for ML WorkflowsMatthew Topol · Øredev 2023
• 27mApache Arrow - The columnar format! What's that?Raúl Cumplido · PyCon US 2024
• 39mTaming the language border (...)Uwe L. Korn · Berlin Buzzwords 2019
• 28mHigh speed data from the Lakehouse to DataFrames with Apache ArrowJim Dowling · PyData Global 2023
• 31mThe Arrow revolution in pandas and DaskPatrick Hoefler · PyData NYC 2023
• 42mColumnar Storage: Redefining Data Management for the Modern EraZoe Steinkamp · Øredev 2024
• 24mPolyglot data with python: Introducing Pandas and Apache ArrowRobson Luis Monteiro Junior · PyCon US 2020