How Google Assistant's architecture powers voice features in your apps

on YouTube ↗20m

" summary

Overview of how Google Assistant maps voice to app actions: speech is streamed to backend with speech biasing to improve transcription; NLU extracts intents and slots, supports multi‑turn dialogue and followups; Android apps plug in via App Actions with capability tags and shortcuts.xml (static shortcuts, inline inventory) so Assistant emits Android intents. Platform supports 66 Built‑in Intents, brandless queries, app install suggestions, and expands triggers for Auto and WearOS.

AI-written summary · about

" chapters (9)

1. 0:00Introduction to Assistant
2. 1:47Vision and Capabilities
3. 3:42Technical Architecture
4. 5:25Speech and Understanding
5. 7:47Context and Dialogue
6. 10:47App Actions Framework
7. 13:21Implementation Details
8. 16:38New Platform Features
9. 18:16Recap and Resources

" related talks

• 20mBuilding Voice Apps for the Google AssistantYuksel Tolun · DevFest Gorky 2019
• 15mDeveloping for Assistant across devicesJessica Dene Earley-Cha, Toni Klopfenstein · Android Dev Summit 2022
• 27mSession 1: Connect your Android app to Google AssistantAdam Coimbra, Prachi Gupta, Shikha Seghal, Michael Puriz, Garrett Gaudini, Megan Cummins Grace · Google Assistant Developer Day 2020
• 16mHow to voicify your Android appMatt Carroll, Shikha Sehgal · Google I/O 2021
• 13mKeynote Interview with Google's Rebecca NathensonRebecca Nathenson · VOICE Global 2021
• 27mAndroid & Assistant: a match made in heavenEliza Camber · droidcon NYC 2019
• 24mDemystifying the Google Assistant for Android developersNicolas Telera · Android Makers 2020
• 27mGoogle Assistant for Android developersElaine Dias Batista · Android Makers 2019