Operationalizing GenAI Applications with DevOps Practices
summary
A content-platform prototype adds summarization, quiz generation and semantic search using RAG with OpenSearch vectors, LangChain/LlamaIndex orchestration and model swaps between hosted APIs and local Hugging Face models; forcing JSON and prompt chains proves brittle but workable, caching cuts token costs, CI GPU tests and per-user sandboxing (Firecracker/E2B) cause most operational pain. Testing uses exact matches, semantic distance and LLM-as-judge; observability tracks time-to-first-token and prompt tracing while an AI platform and enablement teams handle governance.
AI-written summary · about
chapters (13)
- 0:00Introduction and background
- 1:57Building GenAI applications
- 8:02Model selection and experimentation
- 11:55Working with agents and memory
- 15:24Orchestration and frameworks
- 17:09Testing GenAI applications
- 19:34Deploying to production
- 23:03Monitoring and observability
- 25:51Team collaboration and scaling
- 27:42AI platform and enablement
- 30:12Earning trust and adoption
- 31:36Lessons learned and future outlook
- 32:41Q&A and closing remarks
related talks
31mRethinking DevOps and DevSecOps with Generative AI: Insights
24mAIGenOps: Integrating Generative AI into Platform Engineering
33mSimplifying Generative AI App Development Why Standards Matter
40mOpening Keynote: AI & Platform Teams
40mRolling out Enterprise AI: Tools, Insights, & Team Empowerment
15mChallenges in moving Gen AI apps to production
47mEmbedding AI: Don’t Burn Your Fingers in the Heat of the AI revolution- 33mSimplifying Generative AI App Development Why Standards Matter