Systems @Scale Summer 2023
2023-07-18..2023-07-19
-- online --
[Systems @Scale Spring 2021, Systems @Scale Summer 2021, Systems @Scale Fall 2021, Systems @Scale Winter 2021, Systems @Scale Summer 2022, Systems @Scale Winter 2022, Systems @Scale Summer 2023, Systems @Scale 2024]
as the organisers put it
Join leading engineers at Experience Systems @Scale Summer 2023 for insights on managing large-scale information systems.
| title | author | length |
|---|---|---|
| Welcome Day One | Hui Lei | 04m |
Two-day conference program focuses on building and operating planetary-scale distributed systems with a clear agenda: today features a keynote on Ray plus… | ||
| Keynote Q&A | Ion Stoica, Hui Lei | 11m |
Ray splits into a core plus libraries model: use the libraries for common ML workflows, but implement novel distributed algorithms or custom hardware support… | ||
| Ray, a Unified Distributed Framework for the Modern AI Stack | Ion Stoica | 22m |
Ray presents a unified distributed framework that runs data ingestion, preprocessing, training, tuning and serving within one system, exposing only six core… | ||
| System for Scaling Large Language Models | Manisha Jain | 19m |
Describes production lessons for serving multi‑terabyte LLMs: uses LoRA fine‑tuning to avoid full‑model forks, deploys NCCL topology‑aware collectives and CUDA… | ||
| Michelangelo ML Platform at Uber: Past, Present and Future | Min Cai | 19m |
Michelangelo begins in 2016 and now combines a code-first engine (Canvas, launched 2020) and a unified UI (MS Studio) with a Kubernetes-style unified API… | ||
| IPnext: Next Generation Inference Platform | Hitesh Khandelwal, Chao Xie | 16m |
IPnext is a modular control plane with distinct modules—metadata, planner, and deployment—that turns a model’s deployment graph into region- and… | ||
| AI Platform Q&A | Ayichew Hailu | 18m |
Uber’s generative AI platform plugs in multiple detectors — self check GPT, Nemo Gotta Raya framework and a module in Nama index — and reports mixed results on… | ||
| Global Capacity Management with Flux | Hayley Russell, Richard Cornew | 20m |
Flux uses RPC tracing to attribute backend capacity to products, feeds those attributions into a joint mixed-integer programming solver that simultaneously… | ||
| The Evolution of High Performance Networks: A Storage Perspective and Practices | Shu Ma | 18m |
Storage network stack evolution at Alibaba Cloud moves from kernel TCP to Luna user-space TCP for front-end and RDMA for backend, then to a UDP-based Solar… | ||
| ServiceRouter: Hyperscale Service Mesh | Harshit Saokar, Margot Leibold | 21m |
ServiceRouter is Meta’s production service mesh using an embedded routing library (not sidecars) that routes O(10^10) requests/sec across O(10^4) services with… | ||
| Hyperscale Infrastructure Q&A | Ayichew Hailu | 14m |
Live Q&A details operational fixes and numbers: dynamic configuration and kill switches plus a company-wide push for push rule that keeps binaries under 60… | ||
| PolarDB: Alibaba Cloud-Native Relational Database Service | David Zhang | 21m |
PolarDB describes a three-tier, compute‑memory‑storage disaggregated architecture deployed on Alibaba Cloud (500,000+ cores, 80+ AZs) that adds an extra… | ||
| AI Observability At Meta Scale | Riham Selim, Valentin Andrei, Hao Wang, Lei Tian | 22m |
Scalable observability infrastructure at Meta collects host and GPU metrics across 21 data centers using daemons like dynalog and fleet profilers such as… | ||
| Stateful Web Service: A Distributed Lambda Function for Client-Server Interactions | Pouya Zadkhast, Vahid Jazayeri | 20m |
Meta builds a stateful engine called BladeRunner that emulates long‑lived PHP services by keeping connections and subscriptions in a stateful process while… | ||
| Performance Management Q&A | Ariane Jansen | 15m |
PolarDB reports one-sided RDMA writes of about 4–6 microseconds at 4KB and an experiment showing roughly 44% lower cost per query by cutting required cores… | ||
| Service Weaver: A Framework for Writing Distributed Systems | Robert Grandl | 19m |
Service Weaver is a Go framework (serviceweaver.dev) that compiles one single binary deployed as microservices and uses code generation plus a runtime to place… | ||
| Prodspec & Annealing: Intent Based Deployment | Pierre Palatin | 21m |
Google moves most production to Prodspec & Annealing, an intent-first deployment stack started in 2015 for Gmail and now used across many services, claiming… | ||
| The Meta Thrift Journey: O (100 Billion QPS) Moved | DO HYUNG (DAVE) KWON, TJ Yin, Nandhini Santhanam | 20m |
Thrift at Meta handles roughly 100 billion QPS and the team unifies multiple custom RPC frameworks into one fork (FB Thrift) so thousands of machines move from… | ||
| Developer Experience Q&A | Ariane Jansen | 17m |
Service Weaver claims 9x infrastructure cost reduction on a port of Online Boutique (11 microservices), attributing half the gain to runtime collocation and… | ||
| Closing Remarks | Hui Lei | 06m |
Closing Remarks | Hui Lei | ||
| nothing matches | ||