Getting LLMs to do what you want: Output controllers
summary
A deterministic method called JSON DeRuLO enforces a JSON schema at inference time by applying token masking and a cheap character‑level filter so the model samples only legal tokens, guaranteeing 100% structural validity with minimal latency overhead. It runs at runtime so schemas change on the fly, avoids fine‑tuning, and integrates with adjacent mode and inference stacks to produce perfectly structured outputs for APIs, agents, and document extraction.
AI-written summary · about
chapters (8)
- 0:00Structured outputs matter
- 1:05Enforcing output structure
- 3:11Prompt engineering limits
- 4:55Fine-tuning challenges
- 5:38Constrained generation ideal
- 6:20How constrained generation works
- 9:51Structured generation in practice
- 10:58Inference time control future
related talks
17m"Taming LLMs with Structured Outputs"
24mLLM Inference Optimization From Simple To Sophisticated
30mTaking Control of LLM Outputs: An Introductory Journey into Logits
42mRDF and the future of LLMs
47mStructured Outputs in OpenAI: Building Reliable JSON-Based AI Integrations
39mThe Future is Fine-Tuned: Deploying Task-specific LLMs
01h 00mFrom LLM Theory to Practical Agentic Implementations
36mLLMs and AI agents demystified