Guidance: Make your Models Behave
summary
Guidance is an open-source Python library (17K+ GitHub stars) that interleaves code and text to control LMs via token injection and masking, skipping predictable tokens to cut latency and cost (claims of 60–70% wall‑clock reductions and up to 85% tokens skipped in examples). It adds primitives like select and substring, runs locally on Transformers/LlamaCPP and reports 100% accuracy on a JSON extraction benchmark; Phi‑3 support on Azure is coming soon.
AI-written summary · about
chapters (9)
- 0:00Introduction to guidance
- 2:25Prompting challenges
- 7:59Behind the inference stack
- 12:26Token injection optimization
- 18:02Masking and selection
- 22:27Benchmarks and performance
- 26:35Programming with guidance
- 32:44Advanced use cases
- 34:47Future outlook and Q&A
related talks
30mLearning to wield large language models
46mBeyond Basic Prompting: Supercharging Open Source LLMs with LMQL's Structured Generation
21mChatGPT, LLMs, and LangChains - A Beginner's Guide
01h 09mCodex presentation & demo
36mLLMs and AI agents demystified
27mDo you know how well your model is doing? Evaluate your LLMs
01h 00mInside GPT – Large Language Models Demystified
48mDeploy large language models responsibly with Azure AI