GestureCoach: Rehearsing for Engaging Talks with LLM-Driven Gesture Recommendations
summary
System fine-tunes LLMs with adapter modules on gesture data from 10 TED talks (about 1,000 annotated regions and 340 unique video clips) to split the task: the emphasis proposal module predicts exact text spans to gesture (outperforming off‑the‑shelf LLMs in a model evaluation, N=30) and the gesture identification module uses retrieval‑augmented generation to pick video examples. Rehearsal UI queues humanoid-avatar cues 4–5 words early, offers top‑3 choices for personalization, and a user study (N=10) shows more diverse, semantically aligned gestures.
AI-written summary · about
chapters (10)
- 0:00Introduction to GestureCoach
- 0:45Problem with novice speakers
- 2:09GestureCoach solution overview
- 3:34LLM limitations for gesture recommendation
- 4:58Gesture database and modeling
- 6:01Rehearsal interface design
- 7:04User study results
- 8:28Future work and summary
- 9:09Q&A audience perspective
- 10:14Q&A closing remarks
related talks
10mSGToolkit: An Interactive Gesture Authoring Toolkit for Embodied Conversational Agents
01h 39mMastering LLMs: From Prompt Engineering to Agentic AI
50mBeyond the Prompt: Evaluating, Testing, and Securing LLM Applications
22mDiscovering natural language commands in multimodal interfaces
26mHow To Win Friends & Influence LLMs (with Prompt Engineering)
33mOpening Keynote: LLMs, between fantasy and reality - the poin...
01h 15m10x Development: LLMs For the working Programmer
45mPanel Talk