Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviors
summary
Policy maps propose a concrete workflow and an interactive tool called Policy Projector that plots LLM input-output cases (from an Anthropic red‑teaming dataset) into an embedding map, lets authors define if‑then rules referencing named concepts, and immediately tests effects (for example, “if violence + graphic details then suppress graphic details”). An evaluation with 12 AI safety experts shows rapid authoring (about 30 minutes of use) and 31 new concepts created beyond an existing taxonomy.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 0:48What are policy maps
- 2:37Map-making metaphor
- 3:18Core building blocks
- 4:02Policy Projector demo
- 6:07Real-world evaluation
- 8:55Key takeaways
- 9:38Future directions
- 10:00Q&A session
related talks
39mThreat Modeling Using LLMs
01h 00mFrom LLM Theory to Practical Agentic Implementations
53mSemantics-based program manipulation in the age of LLMs
28mExplaining Machine Learning Models with LLMs
15mLLMs for Data-Driven Decision Making in Product Development
29mAutomating Privacy Reviews with LLMs: Challenges & Opportunities
11mExploring Consequences of Privacy Policies with Narrative Generation via Answ...
38mNobody Pages the LLM: Engineering Rigour for Vibe Coding