Wrangling Internet-scale Image Datasets
summary
Released dataset Re-LAION-Caption19M is a concrete claim: 19-million images with improved captions created by filtering multibillion-scale metadata, recomputing captions with lava next, and using OCR, luminance and an aesthetic predictor (Kristoffer Schumm) to trim noise. Engineering choices include WebDataset tar packing, Parquet manifests and JSONL append-only writes with temporary-file renames for crash safety, distributed runs across ~100+ workers and ~128 GPUs, centralized logging via Weights & Biases, and automated plus human inspection — the pipeline works and is reproducible.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:26Talk outline and challenges
- 2:31Downloading image data
- 3:55Storing millions of images
- 5:40Image-text data curation
- 7:04Metrics for filtering images
- 8:49Analyzing aesthetic scores
- 10:11Pitfalls of automated metrics
- 12:39Recaptioning images
- 15:50Building structured databases
- 16:31Data processing challenges
- 17:36Parallelizing data processing
- 19:42Ensuring robustness
- 22:08Tracking data lineage
- 24:16Key takeaways
- 25:18Q&A
related talks
37mData Science for Digital Humanities: Extracting meaning from Images and Text
30mData-Centric AI Cookbook-let's prep that data
39mImpacting Global Policy by Understanding Litter Data
28mSculpting Data for Machine Learning V02
31mBuilding a User Maintainable Visualization Dashboard for...
20mPragmatic Deep Learning for image labelling
41mModels Schm-odels: Why you Should Care about Data-Centric AI
55mKeynote – Joseph Sirosh