Software Engineering Methods For AI-Driven Deductive Legal Reasoning
summary
Argues LLMs can be treated as interpreters of natural‑language programs and applies software‑engineering techniques to real statutes: uses Anthropic Claude 3 Opus and Python scripts, computes a taxable‑income example ($192,350) from selected IRS sections, produces coverage traces, and generates targeted examples via coverage‑guided test synthesis. Demonstrates mutation tests that produce counterexamples and a property‑based loop that finds a CPI condition where the standard deduction can decrease. Code and transcripts live on GitHub and use the SARA dataset.
AI-written summary · about
chapters (12)
- 0:00AI-driven legal reasoning intro
- 2:09Legal reasoning as program analysis
- 7:24LLMs for legal interpretation
- 9:08Test coverage for legal statutes
- 10:53Mutation testing legal clauses
- 12:19Property-based testing legal docs
- 15:26Advanced techniques for legal reasoning
- 16:31Q&A: LLM computation
- 21:44Q&A: Prompt engineering
- 24:30Q&A: Legal ambiguity
- 26:16Q&A: Loopholes in legislation
- 29:06Closing remarks
related talks
28mSymbolic Reasoning in telecom oriented LLMs
27mExploring the Mechanisms of LLM Reasoning Capabilities
40mJudgement Day: Benchmarking "Black Box" LLMs With Open Legal Datasets
55mLessons Learned From Implementing Generative AI Solutions
16mBeyond LLMs - The Next Frontier of Machine Reasoning
47mBuilding Python based AI Systems with LLMs
27mCan LLMs do research?
40mThe causality trap: why your LLM sounds smart but thinks shallow