Experimenting with Hardware and AI to Create Dynamic Live Streaming User Experiences
summary
Experiments show concrete prototypes and outcomes: a millimeter-wave human-presence sensor composites or switches cameras after a solder fix and reliability tuning, a sentiment-driven “mood ring” updates LEDs from computer-vision frames and chat, real-time captioning uses Whisper CPP + WebAssembly for local speech-to-text, a DIY PTZ uses an Xbox controller, Arduino and servo and later face-api-driven auto-follow (works but is janky), and a Unity game streams via WebRTC/WHIP with selectable virtual cameras for viewers.
AI-written summary · about
chapters (12)
- 0:00Intro
- 0:23About Todd Sharp
- 2:21Live Video Has Changed the World
- 5:22What Will Tomorrow Look Like
- 7:06AI in ML
- 9:18Demo
- 10:22Sentiment Analysis
- 11:34Speech to Text
- 12:42RealTime Transcription
- 13:45Human Presence Sensors
- 16:10Human Presence Demo
- 17:32Face Tracking Demo
related talks
37mPersonality Is the New Ringtone: Experiments in AI design
20mBuilding the next interface - AI personas who feel as natural as interacting with a human
19mReal-time Experiments with an AI Co-Scientist
54mCreating new experiences for people in a world of intelligent machines
42mMaking Every Game More Accessible with .NET and AI
17mYour realtime AI is ngmi
23mParallel Eyes: Exploring Human Capability and Behaviors with Paralleled First Person View Sharing
17mTechnologies Enabling Real-Time Interaction With Digital-Humans