Mastering DevOps with AI
summary
A production RAG pipeline links new incidents to past cases using E5 embeddings stored in an ElasticSearch vector index (Haystack hybrid search + FAISS for ANN). Indexed ~182k bugs with human-labeled sets of 9k duplicates and 35k “see also” examples. Baseline top-10 recall is ~35%; fine-tuning improves recall by almost 50%, and a learned reranker reshuffles the top results. The internal “find similar bugs” feature exists in beta and reduces duplicate handling and triage time.
AI-written summary · about
chapters (11)
- 0:00Introduction to DevOps with AI
- 2:12Why not use generic LLMs
- 4:00What is RAG and how it works
- 7:13Introducing find similar bugs
- 9:19Focus on the retriever part
- 12:34Understanding embeddings
- 15:27Evaluating bug similarity
- 19:00Fine-tuning the embedding model
- 21:51Improving with reranking
- 24:00Demo of similar bugs feature
- 25:07Closing and resources
related talks
27mOptimizing RAG Systems in Practice: Challenges and Proven Solutions
40mApplying Artificial Intelligence to the Incident Response Function
16mSewing up success even when RAGs unravel
26mAccelerating DevOps with AI
09mThe AI trick that cut AWS's issue triage time by 90% and caught regressions in 2 minutes
07mInterview
46mLeveraging AI and Automation in DevOps
26mBecoming AI engineers