Product @Scale 2024
who it's for
infrastructure engineers, machine learning engineers, ml platform engineers, product managers, mobile engineers, creator tools engineers, site reliability engineers
topics
AI infrastructure and scaling; GPU capacity planning; PyTorch migration and optimization; custom accelerator deployment; distributed inference platforms; model quantization and pruning; on-device model deployment; copilot and multi-agent orchestration; animation model distillation; evaluation, safety and red-teaming
| title | author | length |
|---|---|---|
| Keynote: AI Infrastructure | Rajesh Nishtala | 12m |
Company scales AI infra from CPU fleets to GPUs and custom silicon, runs training jobs at hundreds to low thousands of GPUs historically and now needs tens of… | ||
| Fireworks AI: Go from Hype to High - Value AI | Lin Qiao | 20m |
Argues that smaller, specialized models beat mega-models on latency, throughput and cost, claiming up to 40x lower inference cost (example: 1B tokens/day saves… | ||
| Capacity Planning for AI Training Hardware | Rahul Rai, Genevieve Ho | 21m |
Meta describes mCP as a process that turns product forecasts into firm hardware orders using capacity contracts and a unified resource schema so new… | ||
| AI Generated Animation | Gaurav Sharma | 17m |
Meta’s Animate product generates high‑quality short animations under 5 seconds by combining PyTorch 2.0/torch.compile optimizations (TorchDynamo… | ||
| Productionizing On-device AI Models | Avni Kakkar | 20m |
Built an on‑device cutout tool for Instagram called Squeeze segment anything by compressing a server Segment Anything model with pruning, quantization and… | ||
| AI Studio - Creating and scaling your custom copilots | Nabila Babar | 15m |
Azure AI Studio shows how copilots move beyond simple RAG into multi-agent systems that combine LLMs, metaprompts, functions and APIs, and it demonstrates… | ||
| Shipping Faster: Behind the Scenes of How We Built | Kathryn Middleton, Chris Li | 21m |
Team builds Meta.a in one month by recruiting platform experts from authentication, web platform, messenger storage and core infra, running parallel prototypes… | ||
| Fireside Chat | Khushboo Taneja, Joseph Laria | 23m |
Meta describes building AI Studio as a no-code platform for anyone to create custom AIs, and reports early wins: a surge in playful creator-made AIs, real-time… | ||
| Experience Creation at Scale Leveraging AI | Stef Corraza | 17m |
Roblox reports 15 million experiences, 70 million DAU and $700M paid to creators and uses AI to cut tasks that take days down to minutes. Features include… | ||
| Write with AI | Joena Zhang | 21m |
Facebook launches Write with AI on Feb 5, 2024, using LLaMa3 (after early tests on LLaMa 270B) to offer real-time editing “pills” that fix grammar, shorten or… | ||
| Talking to others for you: Leveraging machines that speak the “human conversation” protocol | Yoav Tzur | 21m |
Demonstrates building production systems that place machines on real phone calls using Google Duplex, shifting from RNNs to end-to-end LLMs and running… | ||
| Opening Remarks | Sreeram Chakrovorthy | 03m |
Conference frames AI-driven innovation as the theme, cites contributions from Google Microsoft Roblox Fireworks aai and Meta, and claims the technologies… | ||
| Live Q&A | Joseph Laria | 23m |
Meta-style infra practices get applied to ML: resilience, incident management, clear component contracts and efficiency; new needs include batch GPU… | ||
| Closing Remarks | Sreeram Chakrovorthy | 01m |
Conference summary states that sessions catalog the latest advances in AI and concrete examples of AI-enabled product experiences and how companies integrate… | ||
| nothing matches | ||