How We Built Zeta2: Training an Edit Prediction Model in Production
summary
Claims Zeta2 matches frontier quality enough to replace expensive teacher runs, cutting a million frontier requests for 100,000 examples to student model runs 50x at negligible cost. Pipeline uses opt‑in production edit traces, distills with a frontier teacher, applies a repair step for bad outputs, stores giant JSONL records, filters by Levenshtein distance to settled state, and surfaces a reversal ratio and kept‑rate/acceptance dashboards. The system targets the mid‑distance examples and reports offline evals and production metrics.
AI-written summary · about
chapters (5)
- 0:00Introduction to edit models
- 1:04The model training pipeline
- 3:55Utilizing settled data
- 7:02Offline model evaluation
- 7:56Production and experiments
related talks
30mProductionizing an unsupervised machine learning model to...
Stop Paying for Frontier Models
31mFrontier results, on device
18mReality is not an end-to-end prediction problem
38mFrom Experimentation to Products: The Production Machine Learning Journey
31mLarge Scale CTR Prediction Lessons Learned
25mContinuous ML Applications in Production
27mHow to make ML teams 80% more productive.