PyData London 2024
2024-06-14..2024-06-16
London, United Kingdom
[PyData London 2016, PyData London 2017, PyData London 2018, PyData London 2019, PyData London 2022, PyData London 2024, PyData London 2025, PyData London 2026]
who it's for
data scientists, machine learning engineers, data engineers, MLOps engineers, platform engineers, open source maintainers
topics
time series backtesting metrics; generator-coroutines in Python; PyMC probabilistic programming; test-driven data analysis tooling; Prefect workflow pipelines; Polars plugin development with maturin; retrieval augmented generation with vector stores; LLM function calling for structured queries; GPU development for Python; Dask DataFrame 2.0 comparisons; online machine learning with River; observability for Dask in production
| title | author | length |
|---|---|---|
| Backtesting and error metrics for modern time series forecasting | Kishan Manani | 33m |
Backtesting is a tradeoff: choose rolling vs expanding window, step size and refitting frequency to balance reliable error estimates against time and memory… | ||
| Are generator-coroutines really the answer? | James Powell | 42m |
Claim: generator-coroutines let code linearize state machines and externalize stopping criteria, but practical costs appear: Python ≥3.6 async/await and yield… | ||
| Probabilistic Programming and Bayesian Computing with PyMC | Fonnesbeck, Wiecki | 01h 26m |
Uses PyMC v5 (with an optional JAX NUTS backend) to fit a marketing regression, shows MCMC diagnostics (4 chains, thousands of draws, warnings from… | ||
| Test-Driven Data Analysis in Python | Nick Radcliffe | 01h 32m |
Claims a concrete methodology plus an open-source Python toolset: the tdda library for discovering constraints from training data, Rexy for inferring regular… | ||
| Empower Your Projects with Prefect's Pipeline Magic | Adam Hill | 01h 25m |
Hands-on workshop shows Prefect 2.19 pipelines that fetch tweets, publish to Upstash Kafka, run spaCy sentiment, and persist results to MongoDB Atlas while… | ||
| How you (yes, you!) can write a Polars Plugin | Marco Edward Gorelli | 01h 29m |
Claims no prior Rust needed and reproduces a Python numeric map in minutes; workflow uses the cookie cutter template, pip install polars and maturin, write a… | ||
| An Introduction to Retrieval Augmented Generation | Dan Gibson | 48m |
Claims RAG lets organisations build chatbots without training LLMs, using retrieval not model fine‑tuning, and the workshop shows that end‑to‑end with code… | ||
| How to Enhance Customer Targeting in Marketing | Hajime Takeda | 38m |
Shows concrete claims and code: uses CausalML with an XGBoost T‑learner and an uplift random forest demo to estimate effects, finding treatment conversion… | ||
| Enabling real-time insights through stream processing in Python | Adam Glustein | 32m |
Shows actual tooling and results: uses CSP (C++ core with Python API), Bitewax, Flink and Kafka comparisons, GTFS/GBFS feeds and Parquet playback. Builds an… | ||
| A practical guide to human-in-the-loop distillation | Ines Montani | 34m |
Describes a concrete workflow: prompt a large LLM, parse its text to structured output, have humans correct the outputs with human-in-the-loop distillation… | ||
| Multimodal Deep Learning in the Real World | Issac Godfried | 01h 23m |
Claims that multi‑modal models are ready for production meet concrete caveats: OCR + Gemini notebook works for simple scanned pages and returns usable… | ||
| Function Calling for LLMs | Jim Dowling | 35m |
Claims that Retrieval Augmented Generation can query tabular warehouses and APIs by using function calling returns JSON to pick and invoke SQL-like functions… | ||
| Building the composable Python data stack with Kedro & Ibis | Datta, Rodríguez | 01h 24m |
Claim: Python can replace SQL for scalable transformations by using Ibis + Kedro to build a pipeline locally and flip a config to run on a cloud warehouse. The… | ||
| Generating embeddings for Yu-Gi-Oh Cards with NumPy | Antonio Feregrino | 30m |
Builds embeddings from 13,000 cards and >100,000 scraped decks into a 13,000×13,000 cooccurrence matrix (about 5% nonzero), decomposes it with svds SVD to 50… | ||
| GPU development in Python 101 | Andy Terrel, Jacob Tomlinson | 02h 14m |
Claims: Python can write GPU code no C++ required and run it on Colab T4s. Tools actually used: Numba kernels, cuPy, cuDF (the cuDF pandas plugin), Dask with… | ||
| Generating Customers Insights with Topic Modelling and HuggingFace SetFit Method | Sultan Al Awar | 01h 19m |
Hands-on pipeline shows LDA topic modelling (gensim and sklearn) on a Kaggle British Airways reviews dataset, then fine-tunes a HuggingFace… | ||
| What a serverless database means for users | Alex Owens | 30m |
Compares ArcticDB, Parquet (single-file and partitioned) and HDF5 on concrete workloads: 2.5 million rows historical load, 500-row daily appends, single-day… | ||
| Mistakes were made: Data science ten years in | Rebecca Bilbro | 40m |
Claims a low deployment success rate and lists concrete anti‑patterns: building diagnostics like Yellowbrick, chasing t-SNE vs UMAP visual signal, relying on… | ||
| Dask DataFrame 2.0: Comparison to Spark, DuckDB and Polars | Patrick Hoefler | 42m |
Reimplemented Dask DataFrame claims roughly 20x speedups on representative TPCH queries by combining Arrow strings, a new shuffle algorithm and a query… | ||
| Graph Database and Retrieval Augmented Generation | Richard Kehinde Ogunyale | 01h 17m |
Builds a RAG pipeline that actually stores PyData submissions in Neo4j, indexes a document node into a vector store, then runs both vector retrieval and… | ||
| How to Avoid structural biases in evaluating your Machine Learning/NLP projects | Sofie Van Landeghem | 39m |
Assembles concrete evaluation details: mines Wikipedia links, prunes the KB to 14% of concepts, 84% coverage, trains an entity linker with Spacey and scores… | ||
| RAG for a medical company: the technical and product challenges | Noe Achache | 29m |
A RAG for the Vidal is built to let doctors query drug monographs and it uses Chainlit with a vector-backed pipeline over Qdrant and Langsmith. Naive… | ||
| Synthetic Data in Financial Time Series | Stamatis Lykos, Elena Chatzimichali | 27m |
Generates a TimeGAN-style pipeline using RNNs to produce synthetic financial series and an autoencoder-based anomaly detector, trained on WTI and Brent crude… | ||
| The Art and Science of Deep Face Recognition in Python | Sefik Ilkin Serengil | 30m |
Hands-on summary: a ready-to-use DeepFace Python library that wraps ten face models and multiple detectors and exposes a simple verify() API; the project… | ||
| Moving from Offline to Online Machine Learning with River | Tun Shwe | 31m |
Claims River is a practical, pure‑Python library that lets models predict one / learn one on streaming data, lowering CPU and memory by updating on each… | ||
| Linear Programming for Resource Allocation | Carlos Samey | 26m |
A linear program models open days versus mobile clinics across 39 districts and up to four quarterly timepoints, turning an intractable 2^312 brute force… | ||
| 10 years of Parquet- what's next? | Nick Gates | 28m |
Parquet shows concrete wins and clear gaps: a prototype Vortex implementation yields 15% improved compression and roughly 50× faster random access (colleague… | ||
| How continuous testing keeps your LLM on track | Emeli Dral | 41m |
Argues that continuous testing catches regressions and drift by combining an evolving golden dataset, runtime logging, and scheduled checks; shows practical… | ||
| Real-time AI Lakehouse | Lex Avstreikh, Raymond Cunningham | 01h 10m |
Claims a working real-time AI lakehouse built on HopWorks: code uses the BestTime API for foot-traffic, HopWorks feature store, model registry and KServe-style… | ||
| No More Raw SQL: SQLAlchemy & ORMs | Rhythm Patel | 32m |
A Marketplace microservice with five tables (customers, addresses, orders, order_items, items) moves from raw sqlite3 SQL to SQLAlchemy core and then… | ||
| Growing user engagement with RL-driven personalisation | Chris Wilkin | 39m |
Tesco builds a real‑time personalization pipeline using contextual bandits over autoencoder user embeddings, implementing a LinUCB policy with 15-minute batch… | ||
| Building Multi-Agent Generative-AI Applications with AutoGen | Colombo et al. | 34m |
Shows AutoGen open-source Python framework wiring a chat manager, assistant and user-proxy so agents coordinate to decompose planning tasks, call tools… | ||
| The Evolutionary Saga of Python Packaging | Quazi Nafiul Islam | 30m |
Timeline of concrete fixes not promises: setuptools adds dependency declarations and entry points; pip adds uninstall and HTTPS to PyPI; wheel standardizes… | ||
| When and how to start coding with kids | Anna-Lena Popkes | 31m |
Claims concrete ages and tools: human brain has 100 billion neurons and ~2,500 synapses per neuron, reaches ~25% adult size at birth and ~80% by age four… | ||
| Uncertainty estimation at scale with functime | Luca Baggi | 45m |
functime claims to produce prediction intervals for thousands of series locally by combining Polars-powered feature extraction and cross‑validation that runs… | ||
| Adventures in not writing tests | Andy Fundinger | 27m |
Claim: Hypothesis ghostwriter generates runnable tests from type hints so many data-analysis checks get created automatically. Demonstrations run on a… | ||
| Building A Folk Music Recommendation System with LLMs | John Sandall | 40m |
Shows a concrete pipeline: embed 46,000 folk tunes with OpenAI text-embedding-3-small, cluster (KMeans and variants), visualize with UMAP, and validate… | ||
| Using Zarr for drug discovery datasets in Polaris | Cas Wognum | 33m |
Polaris uses Zarr as the canonical store for ML-ready drug discovery data, claiming one format handles kilobytes to terabytes and modalities from SMILES to 3D… | ||
| Protein folding and what it means for drug discovery | Emlyn Clay | 29m |
Predicts that AlphaFold models approach experimental accuracy (AlphaFold 2 GDT >90), and that AlphaFold 3 adopts diffusion models; runs sequences through… | ||
| Achieving Concurrency in Streamlit with a RQ scheduler | Harriet Yue Huang | 32m |
Claim: Streamlit apps become responsive under concurrent users by pushing long jobs into Redis Queue (RQ) workers while the main app stays interactive… | ||
| Log messages processing using NLP tools | Arkadiusz Trawiński | 27m |
A complete monitoring pipeline uses Drain clustering to turn freeform logs into templates and cluster IDs, then fits a multivariate Hawkes process with an… | ||
| Modelling emotional nuance of musical performance | Wojciech Matejuk | 34m |
Claims a pipeline that models piano nuance from MIDI, using Maestro dataset plus private practice recordings to reach ~200 million tokens and training GPT-2 on… | ||
| Are you ready for MLOps? 🫵 | Jeroen Overschie, Jetze Schuurmans | 26m |
Claim: only about half of AI models reach production and another headline figure says ~87% never do; explanation centers on missing foundations not the idea of… | ||
| Data: Faithful or Traitor? | Matthew Crooks | 35m |
BBC uses 150 billion rows a month of Piano analytics and 34 million signed‑in accounts alongside Chartbeat, BARB panels and national surveys; a random forest… | ||
| The art of building and sustaining successful OSS tools and infrastructure | Tania Allard | 46m |
Argues that project success depends on concrete buyable outcomes not lofty topics: 15 years of experience, concrete examples with package tools like pip… | ||
| Quantum Artificial Intelligence | Andrea Melloncelli | 32m |
Claims a working hybrid quantum-classical neural network for drug-response prediction that reports a 15% accuracy improvement over a purely classical baseline… | ||
| Observability for Dask in Production | Hendrik Makait | 35m |
Shows concrete debugging wins: centralized logging plus a topic-based events stream, a native Prometheus plugin, and performance reports that capture realtime… | ||
| Open Source Leadership: What to Give Away and What to Bring In | Deb Nicholson | 29m |
Rule of thumb: if a task demands roughly 30–40 hours a week or carries legal/security risk, move it to paid staff or contractors; if it’s edifying, optional… | ||
| 5 Things I Learnt from Causing a Cloud Provider Outage | Alexander Darby | 34m |
A team causes an outage across Europe by overloading a cloud data lake and loses access to petabytes needed for threat hunting. The rebuild uses Lambda‑style… | ||
| nothing matches | ||