Building Real Time AR experiences at Scale on Constrained Devices
summary
Claim: run GPU-backed ML inference and 3D rendering on nearby edge servers to hit sub-200 ms SLA for shared AR calls and avatar rendering. Uses MetaSpark as a distributed AR engine with PyTorch, CUDA and TensorRT/AI Templates, plus a GStreamer-like media processing pipeline to ingest RGB frames, do ML (body/eye/leg tracking, GANs) and stream 720p+ composited views at ~30 FPS. Meta runs ~10 data centers and hundreds of edge nodes to scale.
AI-written summary · about
chapters (7)
- 0:00Start
- 3:17Shared AR Calling Experiences
- 20:58Meta Spark as an Edge Compute Platform Powered by a Scalable Media Processing Framework
- 23:19Render Frame Rate
- 25:18Low Latency Transport
- 27:39Optimizations for GPU ML Inference
- 29:01Scalable Edge Discovery & Routing
related talks
25mBuilding a Cross-platform Runtime for AR Experiences
15mBeyond Hype: Mawari’s Real-Time Immersive Network
13mCloud XR Experience on 5G with Mobile Edge Networks
22mBringing RTC to Metaverse
41mComputer Vision for Remote AR
45mAzure Remote Rendering
50mAdvanced Mixed Reality Development and Best Practices
38mDesigning Augmented Reality Systems for Speed