The Art of Compression: Crafting Insightful Summaries with LLMs
summary
Project replaces manual handover notes with an on‑prem summarization pipeline that chunks long support conversations, runs map-reduce (parallel) summaries, and enforces a fixed template with assertion checks. Uses self‑hosted LLMs (Mistral, Granite), orchestration with LangChain, fine‑tuning and synthetic training with InstructLab, evaluation by ROUGE/BERTScore and LLM QA/G‑Eval (Mixtral for evaluation), deploys on OpenShift/RHEL AI, and reaches production through staged rollouts in about three months.
AI-written summary · about
chapters (10)
- 0:00LLM summarization introduction
- 3:33Support case summary problem
- 5:20Commercial vs open-source LLMs
- 7:48Document chunking strategies
- 10:35Map reduce vs refine summarization
- 13:23Fine-tuning with InstructLab
- 16:29Evaluating summarization quality
- 24:40Deployment stack and lifecycle
- 28:08Key takeaways
- 29:31Q&A
related talks
13mIntegrating Large Language Models into Security Incident Response
33mDesigning LLM Applications for Massive Datasets
11mLightning Talk: Summarizing the Noise: LLM Observability With Open Data Hub, VLLM...
28mA community based open source LLM and what it can do for your business
33mOpening Keynote: LLMs, between fantasy and reality - the poin...
14mValidation Pipeline for Summary Evaluation
37mExpanding the Capabilities of LLMs Through Community Collaboration
10mSteering Semantic Data Processing With DocWrangler