Building Conversational AI Agents at Scale
summary
ElevenLabs shows a full pipeline that transcribes speech, routes to an LLM, hits a document RAG knowledge base and returns speech with assigned voices per language, claiming live switching across English Chinese Malay and Tamil with both automatic language detection and explicit switch commands. Integrations include uploading URLs and files as a knowledge base, enabling function calling and plugging in custom LLMs such as Gemini 2.0 flash. The demo actually switches languages and voices on the call.
AI-written summary · about
chapters (9)
- 0:00Multilingual conversational AI intro
- 1:28Setting up voices in ElevenLabs
- 3:35Creating new AI agent
- 4:58System prompts and variables
- 6:26Knowledge base and tools
- 7:30Voices per language
- 7:52Live demo of AI agent
- 9:47Language switching modes
- 10:07Closing and call to action
related talks
01h 02m[Full Workshop] Building Conversational AI Agents
17mBuilding Effective Voice Agents
01h 04mBuilding Expertise Into Generated Conversations
12mReal-time voice agents with ElevenLabs and AI SDK
08mGive Your Chat Agent a Voice
47mBuilding sophisticated conversational AI experiences
32mBuilding LLM Voice Bots with Open Source Tools
21mTalking to others for you: Leveraging machines that speak the “human conversation” protocol