Towards Clinically Grounded LLMs
summary
Fine-tuning with LoRA adapters on heavily de-identified real therapist turns shifts baseline LLM behavior toward human therapists across understanding, guided discovery, pacing and collaboration. Training uses prioritized high‑value turns, synthetic user profiles and a frontier judge LLM with an eval suite (MindEval, VeraMH, adapted CTRS and memorization checks). Results show the fine‑tuned model calls 98 techniques versus 68 for base models and exhibits reduced sycophancy activation; a pilot rollout is planned later this year.
AI-written summary · about
chapters (17)
- 0:00Introduction to talk
- 1:50AI and psychology history
- 3:43Sondermind’s AI companion
- 5:09Mental health AI use cases
- 6:13Macro and micro failures
- 9:22Fine-tuning necessity
- 10:24Choosing LoRA for fine-tuning
- 12:54Training process overview
- 14:39Evaluation framework
- 16:46Clinical quality metrics
- 18:34Therapeutic inclination results
- 19:15Technique repertoire analysis
- 20:59Reducing psychopancy failures
- 23:06Lessons learned
- 24:32Ray Summit relevance
- 25:35Future deployment plans
- 25:59Q&A session
related talks
29mLocal LLM fine-tuning: a practical example
38mMaking LLMs reliable - A practical framework
29mCustomizing and Evaluating LLMs, an Ops Perspective
44mProduction-Grade LLMs & AI Agents
31mCustomizing LLMs: A Guide to local Fine-Tuning...
39mHallucination-Free LLMs: Strategies for Monitoring and Mitigation
26mReducing Hallucinations and Evaluating LLMs for Production
28mTrustworthy LLMs: Vibe checks are not all you need