Software Architecture for Gen AI
summary
Argues against building private foundational models because of energy and data costs and recommends RAG and fine-tuning-light patterns instead; warns hallucinations are inherent and systems need human-in-the-loop checks, observability and latency engineering. Recommends choosing public vs on‑prem LLMs, using microservices or serverless for integration, pushing models to the edge deployment for privacy, and running LLMOps for deployment, monitoring and red‑teaming. Notes only about 7% production success for 2023 POCs and suggests simple one‑model-checks-other patterns before multi‑agent complexity.
AI-written summary · about
chapters (28)
- 0:00Introduction to Gen AI impact
- 1:26Hype vs reality in Gen AI
- 3:14Training vs serving AI systems
- 5:45Building on foundational models
- 7:51Security and privacy challenges
- 9:16Observability and fitness functions
- 10:40Hallucinations as a feature
- 13:54Prompt sensitivity and human oversight
- 16:00Non-determinism in AI responses
- 18:56Hardware and infrastructure for Gen AI
- 21:28Where to run Gen AI models
- 23:20Specialized models beyond text
- 25:30Model selection and fine-tuning
- 27:15Open source vs closed source models
- 29:24Synthetic data concerns
- 31:57Future of model improvements
- 33:24Tools for Gen AI development
- 35:33Applications of Gen AI
- 38:04Democratization of AI
- 40:50Sample Gen AI applications
- 45:03Software architecture patterns
- 47:09Retrieval augmented generation
- 51:25Multi-agent systems
- 54:40Key ilities for Gen AI
- 55:43From POC to production
- 57:51Impact on software architects
- 59:15Testing and validation methods
- 1:00:59Future of software delivery
related talks
43mThe Evolution of Design Patterns in the Age of AI
12mBeyond AI Hype: Architecting Software Systems That Scale in the Agentic Era
01h 02mArchitecture Patterns for AI-Powered Applications
11mClean Architecture and AI
24mSoftware 3.0 and the AI Engineering Landscape
11mOpen source GenAI vs. enterprise GenAI: Navigating the future of AI innovation
31mPanel: The AI Architecture Debt—Refactoring Infrastructure for Sustainable Privacy
16mBuilding Applications with AI Agents