Keynote: Designing LLM Native systems
summary
Argues that chatbots often harm products and recommends avoid building chat interfaces in most cases, instead using structured outputs (Instructor library, Ecto schemas) and vector DBs like Chroma to turn unstructured notes into reliable data. Recommends building domain‑specific languages/business‑as‑code for marketing and ops, build your own evals to compare models, and design systems that improve with smarter models, longer context, and faster inference. Gives demos of agent-to-agent binary protocols and warns against overreliance on LLM code generation.
AI-written summary · about
chapters (15)
- 0:00Welcome
- 2:51Speaker background
- 6:20Why LLM native systems matter
- 8:07Chat interfaces limitations
- 12:20LLM economic pyramid
- 14:45Rules for LLM products
- 20:42LLMs in workflows
- 25:59AI assistance vs critical thinking
- 31:56Domain-specific languages
- 32:38Career impact of LLMs
- 35:28Q&A: Community concerns
- 37:36Q&A: Architecting LLM systems
- 38:19Q&A: Elixir code completion
- 39:41Q&A: AI data pollution risks
- 41:05Closing remarks
related talks
17mArchitecting LLM-Based Chatbots using free open-source LLMs
33mOpening Keynote: LLMs, between fantasy and reality - the poin...
38mNobody Pages the LLM: Engineering Rigour for Vibe Coding
54mKeynote: Can you trust your (large language) model?
01h 39mMastering LLMs: From Prompt Engineering to Agentic AI
40mKeynote
44mKeynote Speaker - Simon Willison
10mMastering LLM Chatbot Testing: Metrics, Methods and Mistakes to Avoid