PyData NYC 2024
2024-11-06..2024-11-08
New York, United States
[PyData NYC 2018, PyData NYC 2022, PyData NYC 2023, PyData NYC 2024]
| title | author | length |
|---|---|---|
| Turning DataFrames into Pretty Pictures with Plotnine | Janssens, Chow, Nieuwdorp | 01h 26m |
Demonstrates that plotnine (Python ggplot2) lets one go from verbose plotting code to a four‑line Anscombe plot, claims it covers 95% of ggplot2 use cases, and… | ||
| Explaining Machine Learning Models with LLMs | Avi Levin | 28m |
Claims that LLMs can explain both interpretable models and black boxes by taking model components, producing specific outputs, and finding real issues: uses… | ||
| End-to-end data science with the Positron IDE | Isabel Zimmerman | 21m |
Positron is a VS Code fork for multilingual data science that claims to replace stitching tools by offering an interpreter selector (conda mamba pyenv venv… | ||
| Building Women in Data Communities: A Journey of Empowerment | Borar et al. | 39m |
Community organizers describe concrete outcomes: a 10-session reading group that keeps meeting, a new member-run community called Women who do data with active… | ||
| Using NASA EarthData Cloud & Python to Model Climate Risks | Dhavide Aruliah | 01h 18m |
Hands-on pipeline that actually logs into NASA EarthData Cloud (netrc credentials), queries the STAC catalog, and stacks remote granules to compute area… | ||
| Mastering Gaussian Processes with PyMC | Alexandre Andorra, Christopher Fonnesbeck | 01h 33m |
Shows how to specify, fit and validate Gaussian processes in PyMC with concrete code and notebooks: HSGP approximation for speed, sparse inducing points for… | ||
| Unlocking the Power of Hybrid Search: A Deep Dive into Python-Powered Precision | Art Anderson | 57m |
A concrete build story: Python glue wires Scrapy + LlamaIndex for chunking, Gemini (after trying Nomic AI) for 768‑dim embeddings, and Aerospike as a NoSQL… | ||
| PyMC-Marketing: Customer and Marketing Analytics the Easy Way | Christian Luhmann | 41m |
An open source library called PyMC-Marketing delivers two concrete modules: marketing mix models that are fully Bayesian on top of PyMC with a PyTensor backend… | ||
| How to Deploy Machine Learning Inference Code to Production | Saba Nejad | 40m |
Code profiling with cProfile finds repeated S3 downloads and 96 per‑sentence model loads, and the pipeline uses torch, SentenceTransformers/E5‑base embeddings… | ||
| Faster PySpark with Apache Arrow | Allison Wang | 29m |
Claims that Apache Arrow reduces PySpark serialization cost and speeds Python UDFs by concrete amounts and shows working examples. Benchmarks show… | ||
| Eliminate Hallucinations with Context Filter Guardrails | Josh Reini | 29m |
Claims that context filter guardrails eliminate hallucination in RAG by scoring each retrieved chunk and dropping those below a threshold, using feedback… | ||
| Adopting Open-Source Tools for Time Series Forecasting | Chowdhury, Murthy | 32m |
A decision framework evaluates actual capabilities not topics: it scores libraries on Data Understanding, Data Preparation, and Modeling & Backtesting and… | ||
| Get insights from structured and unstructured data using the BigQuery DataFrames package | Tim Swena | 01h 08m |
BigQuery DataFrames demonstrates pushing pandaslike operations into BigQuery, implements over 700 pandas methods, and runs on scalable storage/compute… | ||
| Building machine learning pipelines that scale | A. Datta, D. Datta | 01h 40m |
A reproducible pipeline that predicts a chess live win probability gets built end to end using DT for ingestion, feature engineering in Ibis and Ibis ml… | ||
| An Introduction to Retrieval Augmented Generation | Kumar, Pattaniyil | 01h 14m |
Creates a working Retrieval Augmented Generation pipeline that pulls YouTube videos transcripts and sampled frames, chunks text (token and recursive… | ||
| Advanced Excel Analytics: Python Integration Workshop | Hewitt, Kitchen | 01h 25m |
Hands-on workshop walks through Python-in-Excel plus the Anaconda Toolbox, demonstrates building a working sentiment pipeline (NLTK/VADER) that scores Amazon… | ||
| Run Large-Scale Graph Analytics Using The Most Popular Graph Analytics Library Available | Rick Ratzel | 40m |
NetworkX adds backend support and an NVIDIA backend nx-cugraph implements cuGraph so existing NetworkX code runs on GPU by setting an environment variable… | ||
| Unstructured Data Processing with a Raspberry Pi AI Kit and Python | Timothy J Spann | 37m |
A portable pipeline captures webcam frames on a Raspberry Pi 5 with a $70 AI hat, runs object detection and pose estimation (ResNet34/pose nodes), vectorizes… | ||
| Preparing data for LLM training with Data Prep Kit | Santosh Borse | 40m |
Claims a production pipeline and an open source tool called Data Prep Kit (DPK) that actually turns Common Crawl into training-ready Parquet, using PDF/HTML… | ||
| No More Raw SQL with SQLAlchemy and ORMs | Rhythm Patel | 37m |
Claim: stop writing raw SQL and let SQLAlchemy replace it with programmatic queries that are shorter, safer, and easier to maintain. Uses engine and session… | ||
| Easy Model Blending using Pseudo-Bayesian Model Averaging in Python | Nathaniel Haines | 35m |
Repository releases an open source BayesBlend package that fits candidate models with CmdStanPy/ArviZ, extracts out-of-sample likelihoods and posterior… | ||
| Graph neural networks for biomedical insights | Rohit Tripathy | 40m |
Claims a pipeline that fuses transcriptomics and proteomics with a GNN + Transformer architecture on ROSMAP data (~200 paired samples, 20,000 genes 9,000… | ||
| Domain-driven Data Science | Pietro Peterlongo | 43m |
Argues domain expertise materially improves outcomes by showing concrete work in supply chain: builds a planning product that combines probabilistic forecasts… | ||
| Can You Tell the Truth by Lying? | James Powell | 33m |
Argues that precise truths often hide usefulness so pragmatic simplifications win: uses concrete Python examples — decorators (contrasting the lie decorator is… | ||
| Building Python based AI Systems with LLMs | Jim Dowling | 47m |
Claim: LLMs reliably translate text queries into runnable Python, using Hopsworks feature store and pandas to produce feature, training, and inference… | ||
| Context-Aware Masking for Controlled Music Generation in MIDI | Matejuk, Roszczynialski | 33m |
Preprocessing and masked-infilling methods target MIDI performance data with a 10 millisecond sampling grid, tokenizing velocity, note_on/time increments and… | ||
| Reproducible work environments for data scientists using Nix | Avik Basu | 30m |
Argues that Nix makes development environments deterministic by treating packages as immutable and declarative, demonstrates a small project that uses Nix… | ||
| Building Production-Ready AI Systems in 90 Minutes | Nick Tchayka | 01h 13m |
Builds a working RAG pipeline in 90 minutes using LangChain plus ChromaDB and OpenAI embeddings from the repo; loads a PDF, splits into chunks, generates… | ||
| Building Your First Multimodal Gen AI App 🚀 | Hugo Bowne-Anderson | 01h 30m |
A hands‑on app is built live: a Streamlit front end in GitHub Codespaces that takes speech, transcribes it, passes text to an LLM to make a song, and fans that… | ||
| Deploy & Monitor ML Pipelines with Python, Docker and GitHub Actions | Rami Krispin | 01h 11m |
Shows a reproducible pipeline that pulls New York electricity data for 11 subregions, uses Python inside Docker images orchestrated by GitHub Actions + Docker… | ||
| Vector Databases Demystified: Discover how they work | Art Anderson, Tim Faulkes | 01h 23m |
Shows concrete results not fluff: a runnable repo (create-a-vector-db) that builds an in-memory vector store using all-miniLM L6 v2 384‑dim embeddings, a… | ||
| The Art of Compression: Crafting Insightful Summaries with LLMs | Aarti Jha | 38m |
Project replaces manual handover notes with an on‑prem summarization pipeline that chunks long support conversations, runs map-reduce (parallel) summaries, and… | ||
| Pushing Cython to its Limits in Scikit-learn | Thomas J Fan | 39m |
scikit-learn shows exactly what was built and what worked: Cython + C++ stdlib + OpenMP for parallelism, Tempita code generation, and fused types produce… | ||
| Data preparation and analytics on your laptop with DuckDB | Matson, Prawiroatmodjo | 33m |
Claims are concrete: DuckDB reads a 160 GB compressed eBird file with read_csv and casts messy columns into types, loads into MotherDuck with an initial ingest… | ||
| Density! The not-so-secret trend driving the future of Data Analytics | Benjamin Zaitlen | 39m |
Argues that densifying hardware — packing more CPU cores, H100 80 GB GPUs, higher memory bandwidth and NVLink — changes what problems are practical… | ||
| Daunting to Doable: How Institutions can encourage Open Source Contributions | Kaushik Srinivasan | 22m |
Bloomberg matches contributed open source hours to charitable donations so employees log coding time and trigger volunteer hours matched to donations; eligible… | ||
| An Introduction to Polars | Matt Harrison | 01h 32m |
Compares Pandas and Polars by showing concrete moves: loads a Strava GPX, casts elapsed seconds down to int16 (fails at int8), converts timestamps to Mountain… | ||
| Straightforward stream processing with CSP | Adam Glustein | 39m |
CSP is an open‑source stream processing library with a C++ engine and Python API that aims to keep research and deployment identical: the same graph runs in… | ||
| Use ARTKIT to Automate and Scale Up Your LLM Evaluation Process | Gao, Slater | 38m |
BCG claims ARTKIT automates and scales LLM evaluation and ships as an open-source Python package that wraps vendor and custom models with asynchronous model… | ||
| How many dataframe libraries do you need to change a lightbulb? | Dharhas Pothina | 35m |
Compares pandas, Polars, and DuckDB by concrete limits and wins: pandas fails on a 50 GB dataset on a single laptop while Polars and DuckDB stream and complete… | ||
| Analyzing the Impact of Music, Posters, & Trailer Tonality Using Machine Learning | Niranjan Ganesan | 35m |
Claims movie box office class prediction with 89% accuracy using poster features alone and plans to add trailer tonality scored out of 10. Uses data from TMDB… | ||
| Do Pythons Rust? How we used PyO3 to build a Python Stream Processor w/ a Rust Heart | Zander Matheson | 44m |
Shows concrete wins and tradeoffs: using PyO3/PyOxide plus Maturin compiles Rust into a Python module and a recursive Fibonacci (n=35) example moves from… | ||
| Holistic root cause analysis of software breakages through structural causal modeling | Cole et al. | 38m |
Claims a fast, automated, holistic root cause analysis that uses an ensemble of structural causal models trained with the doy library and assigns contributions… | ||
| Temporal Analysis on Topics Utilizing Word2Vec | Narayan Gupta, Sandhu, Wajid | 39m |
Claims a reproducible pipeline that detects semantic change over time by training separate Word2Vec embeddings per time slice, measuring drift with cosine… | ||
| Data Secrets from a Platform Engineer | Rebecca Bilbro | 38m |
Claim that rewriting ORM queries to raw SQL cuts runtime from 8 hours to minutes; note warehouses lack DB guarantees like global IDs so entity‑resolution… | ||
| Trustworthy AI in Healthcare and Life Sciences | Sabrina Hsueh | 41m |
Claims that biased data shifts a Black patient’s liver transplant waitlist four-year waitlist shift and affects 14,000 patients, plus examples of late… | ||
| Building and scaling AI applications for Data Scientists and Developers | Amanda Silver | 55m |
Live demos show concrete outcomes: a ready-to-run AI app template gallery that scaffolds Python/Flask apps and deploys to Azure with a single cloud command, VS… | ||
| Making OSS AI Real with Knowledge Graphs | Alexy Khrabrov | 42m |
Clear claim: combining Knowledge Graphs with retrieval (the GraphRAG pattern) gives grounded, long‑range context that pure vector RAG misses. Concrete tools… | ||
| The Graph Database of Everything Open: Navigating the Open Science Ecosystem w/ MOSS | Jonathan Starr | 46m |
A live Map of Open Source Science (MOSS) and SOL database that actually links projects papers people organizations and dependencies is under construction and… | ||
| Python and the AI Lakehouse | Javier de la Rúa Martínez | 30m |
Claims that Python becomes native to the Lakehouse by adding a feature store with online sync to operational DBs, a vector index and model registry, and an AI… | ||
| Polars GPU acceleration | Ritchie Vink | 35m |
Polars reaches 1.0 and now wires GPU acceleration with cuDF into the optimizer by inserting a GPU callback after the Polars IR, producing a hybrid CPU/GPU… | ||
| Preparing Data for LLM pretraining using open source Data Prep Kit | Santosh Borse | 01h 30m |
Starts from Common Crawl and other sources and shows actual steps: PDF→par extraction, paragraph chunking, exact and fuzzy deduplication that drops ~60–70%… | ||
| GPU Accelerated Python | McCarty, Riehl, Tomlinson | 02h 28m |
Shows concrete wins and code paths: hands‑on notebooks use Numba kernels, CuPy arrays and cuDF/pandas plugin under CUDA on cloud VMs. Example: a 30M-element… | ||
| Accelerating GPU Algorithms in pure Python | Bryce Adelstein Lelbach, Andy Terrel | 34m |
Claims that pure Python kernels JIT‑compile to the same code and match CUDA C++ performance, demonstrated with a copy, tiled matrix transpose, and batched… | ||
| Open Source, Open Roads: Mapping the Human Element in Tech | Lawrence Gray, PhD | 43m |
Describes a concrete claim that Python rescues a decade-long debilitating bipolar depression unresponsive to 15 psychiatric medications by giving cognitive… | ||
| Lightning Talks | 32m | |
Multiple short projects report concrete results: a contribution to BigQuery DataFrames implements explode/unexplode for struct columns, compiles expression… | ||
| Mastering DataFrame Diffing Techniques | Jinxuan Wu | 39m |
Compares concrete methods and results: uses pandas.testing.assert_equal and join+indicator for small checks, shows duckdb anti-join and SQL queries running on… | ||
| Ibis: Don't Let the Engine Dictate the Interface | Gil Forsyth | 42m |
Ibis is a pure‑Python dataframe interface that hands queries to engines instead of running them locally, fetching only table schemas until execution and… | ||
| Interpretable Anomaly Detection for Numerical Data in Python Using PyNomaly | Constantinou, Tiras | 29m |
PyNomaly implements LoOP to produce 0–1 probability scores for numerical anomalies and ships as a lean Python + numpy library with optional Numba and pandas… | ||
| What we learned by converting a large codebase from Pandas to Polars | Janssens, Nieuwdorp | 39m |
Converted about 20,000 lines from Pandas plus some R to Polars, with concrete benchmarks: isolated hotspots drop from 30s → 1s after switching RDS to parquet… | ||
| Hear what people had to say about PyData NYC! | Don Wages, Will Dean, Pur Sharma | 02m |
Short on claims and long on scenes this wrapup names concrete activities and outcomes it reports hosted the diversity lunch creating space for Diversity… | ||
| ArcticDB, the OLAP antidote | James Munro | 36m |
Claims that OLAP often hurts data science not helps, and that big cloud OLAP is expensive and slower for downloads than smart client‑side reads. ArcticDB is a… | ||
| nothing matches | ||