Processing Geospatial Data at Scale with Python and Apache Sedona (incubating).
summary
Apache Sedona (incubating) Python API extends Spark 2.4/3.0 to load and process vector data at scale, with concrete examples using Docker notebooks and a NYC taxi + OSM route dataset; it reads geojson, shapefile, WKT, WKB and integrates with GeoPandas/Shapely, supports spatial predicates (distance, intersects, within), spatial joins and a broadcast join, and requires two extra jars (Sedona adapter + GeoTools) plus sedona registrator. It uses kdbtree quadtree rtree partitioning with global/local indexes and custom WKB/shape serialization, claims 2–10× speedups and lower memory, and shows results that run and visualize without extra JVM-to-Python serialization.
AI-written summary · about
chapters (23)
- 0:00Introduction to talk
- 1:11Geospatial data importance
- 2:58Apache Sedona overview
- 6:31Spatial indexes explained
- 8:41Spatial partitioning
- 10:06Optimizing memory usage
- 11:10Sedona Python API
- 13:21Sedona indexes in Python
- 14:24Sedona SQL functions
- 15:49Python RDD API
- 17:18Geospatial DataFrame
- 18:41Installing Sedona
- 20:05Python API history
- 20:50Reading geospatial data
- 23:17Notebook setup
- 25:26Required dependencies
- 26:32Reading data formats
- 29:48Geopandas integration
- 31:15Taxi data preprocessing
- 33:27Spatial join examples
- 35:38RDD API vs DataFrame
- 36:46Broadcast join demo
- 37:07Q&A session
related talks
46mHow Apache Sedona is Revolutionizing Geospatial Data Analysis
29mGeospatial Analysis using Python
31mUnpacking the geospatial engineering toolbox – an overview of data science techniques for spatial da
30mGeospatial analysis made easy with PostGIS and Geoalchemy
01mA Practical Introduction to Geospatial Python
50mBig Data Processing and Data Science Environment in Python
31mOpenDataCube - Petabyte Scale Satellite Imagery Processing in Python
20mOpen-source Python tools to visualize and analyse ... with Samweli Mwakisambwe