Democratizing AI - Back-fitting end-to-end machine learning
summary
LinkedIn rearchitects ML around four concrete layers — Exploring and Authoring, Training, Deploying, Running — and claims doubling modeler efficiency by shipping a Feature Marketplace for reusable, online/offline-consistent features, a declarative Quasar DSL and Photon connector for training and inference artifacts, and automated health checks that trigger retrain-and-deploy when model drift exceeds thresholds. Deployment packs global models to search nodes and stores per-member coefficients in key-value services for real-time ranking.
AI-written summary · about
chapters (19)
- 0:00Intro
- 0:41Al for Linkedin products
- 10:30Goal of Pro-ML
- 11:12Pro-ML Layers
- 13:43Organizationally
- 17:48Exploring and Authoring - Jupyter
- 18:52Exploring and Authoring - Quasar
- 21:09Exploring and Authoring - Spark
- 22:03Features and Algorithms
- 22:33Feature Marketplace
- 24:08Feature Access Consistency
- 29:25Training Engine: Photon-Connect
- 32:58How We Learn From Data
- 34:14How We Make it Happen
- 35:42Health Assurance Model Development Cycle
- 37:50Anomaly Detection
- 38:33Al Academy Accelerated Democratization
- 40:02Adoption
- 42:38Democratizing AI
related talks
27mSimplify and accelerate AI for the entire data science team
38mFantastic ML apps and how to build them
32mTeam Teardown: Evolving an ML Platform
41mInfrastructure for ML Applications
32mRebuilding the AI Platform: Designing for Today’s Chaos and Tomorrow’s Unknowns
36mKeynote: AI, ML, Cloud for ML
19mDeveloper Centric AI
38mArtificial Intelligence and Machine Learning