Beyond Captions A Practical Pipeline for Live AI-Generated Sign Language Avatars
summary
A working pipeline converts live audio into expressive sign avatars injected into an HTTP Adaptive Streaming (HAS) feed by chaining Audio-to-Text, Text-to-Gloss, Gloss-to-Pose, Pose-to-Avatar. The team ships a demo that concatenates recorded gloss examples into skeleton sequences, streams skeletons to clients, and reconstructs avatars with a point‑stream approach; alternatives evaluated include cloud ASR for accuracy, edge Text-to-Gloss for dialects, client GAN/diffusion or server Unity rendering for GPU versus bandwidth tradeoffs; 2.5 million users motivate the low‑latency design.
AI-written summary · about
related talks
46mBuilding accessible experiences with AI
24mSIGN TO TEXT Unleashing the Power of Google AI for real time ASL transcription
30mBuilding a Sign-to-Speech Prototype with TensorFlow, Pytorch, and DeepStack
27mWhat happens when AI starts grocking streaming audio directly
01h 03mIt's All About AI
26mBuilding a Sign-to-Speech prototype with TensorFlow, Pytorch and DeepStack: How...
57mApplied AI and Accessibility to Play Games in New Ways
16mThe End of Awkward AI Transcriptions