Training AI To Code Using The Largest Code Dataset (Project CodeNet)
summary
Project CodeNet is a curated dataset of 14 million submissions solving 4,000 programming problems across 55 languages with over half a billion lines of code and per-submission CPU/memory and verdict metadata. Tools include AST/tokenization, masked-language and token models, Papermill notebooks, Kubeflow/Q4 pipelines, DataStream, and kServe for model serving. Demo trains a language classifier notebook from cached PVC data, reaches near-perfect accuracy on 100-sample tests with one C++ misclassification, and deploys the model as a Flask/kServe container on localhost:5000.
AI-written summary · about
chapters (19)
- 0:00Start
- 7:32Project CodeNet: Polyglot of languages
- 7:48The Data
- 9:03Metadata - Dataset Level
- 9:24Metadata - Problem Level
- 9:58Metadata - Submission status codes
- 10:14Tools and Examples
- 11:27Real-World Applications
- 13:43The Machine Learning eXchange (MLX)
- 16:26Datasets -Integration with Datashim
- 18:51Watson Studio Pipelines Open Beta and CPD Rele
- 19:28Speeding up the Al Lifecycle!
- 21:00Pillars of Al Lifecycle - Datasets, Models..
- 21:41and Pipelines
- 23:03View, download, and execute Pipelines
- 23:54View, download, and execute Pipeline Components
- 24:46Library of prepackaged models. Register your own models, run with Pipe
- 25:25Register and deploy Datasets
- 26:36Library of prepackaged notebooks. Register your own notebooks
related talks
39mTraining AI To Code Using The Largest Code Dataset (Project CodeNet)
37mTraining AI To Code Using the Largest Code Dataset
46mCode + AI: Will Robots Take Our Coding Jobs? ML Applied to Programming
28mDelivering AI on Code: Live Demo of source{d}
27mTeaching a Machine to Code
01h 09mCodex presentation & demo
15mKeynote: Pre-training and Fine-tuning of Code Generation Models
43mMachine Learning for Developer Productivity