Bringing RTC to Metaverse
summary
Meta converts phone calls into avatar sessions that stream skeletal and expression data over WebRTC, using a root-level present service to maintain calls across app/process boundaries. Video avoids copying by using Android textured surface or shared memory; audio and spatialization either send metadata or shared buffers. Photorealistic faces use personalized ML codecs that compress expressions into a 256-byte embedding (under ~50 KB on the wire). World state uses network IDs, delta serialization, UDP with an FEC-like reliability layer; cloud rendering targets ~50 ms round trips for partial offload.
AI-written summary · about
chapters (4)
related talks
19mCloud Streaming in Metaverse
10mRTC @Scale, Future RTC Experiences Session - Live Q&A
32mBuilding Real Time AR experiences at Scale on Constrained Devices
41mRTC in the Metaverse @Scale Live Panel
17mPath to Metaverse-ready Networks
38mReal-time Communication on Mobile
44mMatrixRTC: The (not anymore) missing piece to enable great video conferences
11mKeynote: Real-time Communication for Today and Future Experiences