From Text to Complex Structured Data: When the Prompt Is No Longer Enough
summary
PayFit describes using LLMs’ structured output option to extract payroll parameters from legal text, addressing complex business-schema constraints that a simple prompt can’t guarantee. They combine a pre-programmed feedback loop that automatically validates and corrects LLM outputs against syntax and semantic rules, and a systematic evaluation framework with business metrics to choose models and detect prompt regressions. The talk provides immediately applicable patterns for integrating these techniques into agents or workflows while preserving human expert oversight.
AI-written summary · about
chapters (6)
- 0:00Payroll with LLMs
- 3:15Legal text automation with LLMs
- 13:04Feedback Loop pattern
- 19:56Model evaluation
- 24:31Project results
- 28:43Q&A
related talks
10mSteering Semantic Data Processing With DocWrangler
15mFrom Words to Data: Using LLMs to Turn Questions into SQL
01h 39mMastering LLMs: From Prompt Engineering to Agentic AI
30mIs Prompt Engineering Dead? How Auto-Optimization is Changing the Game
46mBeyond Basic Prompting: Supercharging Open Source LLMs with LMQL's Structured Generation
19mUnlocking the Full Potential of LLMs: Advanced Prompt Engineering Techniques
48mBeyond the Prompt: Evaluating, Testing, and Securing LLM Applications
33mDesigning LLM Applications for Massive Datasets