Live Keynote
summary
Meta describes rapid infrastructure changes: a virtual machine vending machine built in months runs 250,000 concurrent VMs and reports 40 billion+ code executions since January; training uses clusters of 100,000+ GPUs with plans for a million GPUs by year end; distributed inference forces model partitioning, GPU‑to‑GPU routing and new placement constraints; the stack shifts from SPMD to a single controller programming model; mitigations include QLC SSDs, cloud-overlaid regions with Twine, RAG with vector DBs and graph RAG, and handling power oscillations 10–15 MW during large jobs.
AI-written summary · about
chapters (5)
- 0:00Optionality, time-to-market, and innovation are key principles guiding AtScale's infrastructure development.). []
- 2:00Advancements in post-training and reinforcement learning are creating new systems challenges around scaling, efficiency, and latency/reliability.). [04:48]
- 8:00Distributed inference is a growing challenge as models become too large to fit on a single GPU.). []
- 10:00Reducing hallucinations in large language models is a key focus area, leveraging techniques like retrieval-augmented generation.). [10:42]
- 11:00The programming model for AI is shifting from "single program, multiple data" to a "single controller" model to enable more flexible parallelism.).
