Voice and Visual: How to Build Actions for Smart Displays and More
summary
Built a multimodal Virtual Pet game (hamster with badges, energy bar, levels) that runs on speakers, smart displays and mobile by using Dialogflow, Firebase (Cloud Functions, Firestore, Storage) and Actions on Google; it leverages SSML and the free Sound Library for layered audio, Rich Responses (carousel min 2 max 10) for discovery, and Interactive Canvas (HTML/CSS/JS) for touchable animations. The design adds branching dialogs, progressive disclosure and a daily interaction cap to avoid infinite loops, scales across ~1 billion Assistant devices and works end to end.
AI-written summary · about
chapters (10)
- 0:00Introduction to action design
- 1:35Game development tools
- 3:26The virtual pet concept
- 7:22Five pillars of action design
- 10:55Refining voice interactions
- 14:24Voice-first design principles
- 21:53Visual and rich responses
- 29:20Interactive canvas
- 33:29Scaling to mobile devices
- 36:34Testing and resources
related talks
44mDeveloping Skills & Actions for Voice Assistants
33mBuilding Conversational Experiences with Actions on Google
20mBuilding Voice Apps for the Google Assistant
42mWelcome to VOICE: Opportunities and Challenges for Brands
42mProgramming Amazing Voice Interactions
01mUX/UI
32mParty like it's 1970: Conversational interfaces are back (into your Android app)
17mOK Google: Build Interactive Apps with Google Assistant