Dataset enrichment using LLM's ✨
summary
A pipeline extracts hidden housing features from listing text into JSON/Pydantic objects using openai and instructor, compares GPT-3.5 GPT-4 and PaLM on ~100 samples and reports ~60% exact property accuracy for the baseline prompt. Function calling forces structured outputs but performance varies per model (GPT‑4 initially wraps extra text, later corrected), hallucinations occur on minimal descriptions, and the experiment runs for under €100. function calling Pydantic schema
AI-written summary · about
chapters (15)
- 0:00Introduction to dataset enrichment
- 1:24LLMs booming in recent years
- 3:11Information extraction with LLMs
- 5:19Naive LLM prompt approaches
- 8:11Structuring LLM output with JSON
- 10:59Handling LLM hallucinations
- 12:02Evaluating LLM performance
- 13:50Benchmarking LLM models
- 16:20Pydantic schemas for structured output
- 19:11OpenAI function calling for extraction
- 21:18Forcing structured output via function calls
- 24:57Key takeaways
- 27:49Q&A on costs and cleaning
- 31:51Q&A on multilingual challenges
- 33:40Closing remarks
related talks
43mState of GPT
01h 00mGoing beyond ChatGPT: introduction to prompt engineering & LLMs
40mJudgement Day: Benchmarking "Black Box" LLMs With Open Legal Datasets
41mConnect GPT with your data: Retrieval-augmented Generation
01h 21mAgent Optimization with Pydantic AI: GEPA, Evals, Feedback Loops
24mLLMs: Powering Lightning Fast Product Innovation
23mPanel
35mBuilding LLM-based Agents