SGToolkit: An Interactive Gesture Authoring Toolkit for Embodied Conversational Agents
summary
SGToolkit is an interactive authoring system that generates upper‑body gestures from speech using a neural generative model trained on TED data, exposing fine pose edits and coarse style sliders; it uses Huber loss + GAN loss and concatenated pose/style control vectors. A user study (24 designers) shows faster creation under time pressure and 4000+ crowd ratings (120 raters) compare toolkit, manual keyframing, and automatic generation. Code and Blender/web frontends are available on GitHub and the system is platform‑agnostic.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:40Demonstration
- 3:58Model Overview
- 5:55User Study
- 7:21PostExperiment
- 7:47Quality Assessment
- 9:28Summary
related talks
11mGestureCoach: Rehearsing for Engaging Talks with LLM-Driven Gesture Recommendations
13mSketchGPT: A Sketch-based Multimodal Interface for Application-Agnostic LLM Interaction
56mBeyond Chat: Bringing Models to the Canvas
14mDialogLab: Authoring, Simulating, and Testing Dynamic Human-AI Group Conversations
07mSketchar: Supporting Character Design and Illustration Prototyping Using Generative AI
12mAI Content Generation
34mUsing Appsmith for building GenerativeAI apps
06mInterview