Any-to-Any: Building Native Multimodal Agents
summary
Walks through a concrete any-to-any stack: Gemini ingests PDFs, video and audio up to nine-plus hours and uses function/tool calling to trigger native generators like Nano Banana for images and a text-to-speech model for podcast audio the NotebookLM clone is an agentic loop where a reasoning model decides what to generate not a hardcoded pipeline the live API is a single audio to audio architecture with no cascaded pipeline and context caching saves about 90% of repeated multimodal costs and the demo yields text speech and infographics end to end
AI-written summary · about
chapters (14)
- 0:00Introduction to the session
- 0:58Defining "Any-to-Any" and the Gemini ecosystem
- 2:56Building a NotebookLM clone using an agentic approach
- 3:51The agentic architecture for multimodal applications
- 4:50Implementation details for multimodal understanding
- 6:10Tips for audio/video processing and context caching
- 7:56Multimodal generation phase
- 8:37Native image and infographic generation
- 9:04Native speech generation and podcast style audio
- 9:57Implementing function/tool calling
- 11:28The power of native generation models
- 12:37Multi-language and accent capabilities in audio models
- 13:46Live API and real-time interaction
- 15:06Final summary and additional model shout-outs
related talks
37mBuilding Multimodal AI Agents From Scratch
01h 34mHands-on AI workshop: Multimodal agents
30mLook ma, no hands! Multimodal AI-Agents in the browser
33mLeveraging multimodal language model for AI native wireless systems
33mBuilding Cross-Platform, Multi-Modal Agents with Flutter
52mChoose your own adventure in agentic design patterns
29mTools and Roadmap for AI-Native Developers
40mThe Marvelous Magic of Multimodal AI