Using Open Source LLM in ETL
summary
Case study uses Llama2-70b to convert applicant Q&A into features for reinsurance ETL by doing binary disease classification + chaining + regex→JSON, evaluates with precision and recall against underwriter labels, and runs locally with bit‑quantization and inference of 4–6s per call on two A100. Temperature is 0.1. System reliably flags yes/no and unanswerable cases, struggles with numeric extraction, occasional hallucinations or blank outputs, and otherwise produces an industry‑agnostic pipeline that mostly works.
AI-written summary · about
chapters (7)
- 0:00Reinsurance and ETL challenges
- 3:39LLM agents for underwriters in ETL
- 7:57Trustworthiness of LLM outputs
- 14:38Prompt engineering for consistent results
- 16:27Hallucinations and prompt iteration
- 22:17Inference speed and performance
- 27:57Automating underwriting with ML
related talks
32mEvaluation of QA systems built on LlamaIndex
44mPimp My Llama - How to Customize LLMs for your Business Case
01h 49mAI engineering with open access LLMs that lie, curse, and steal
33mDesigning LLM Applications for Massive Datasets
56mDehumanizing Agents: Why Explainability is Crucial in the LLM Era
59mWorkshop: Choosing the Best Open Source LLM for Your Application
48mBringing Llama 3 to Life
46mBeyond Basic Prompting: Supercharging Open Source LLMs with LMQL's Structured Generation