Accelerating Musical Live Coding With On-Device AI
summary
A browser-based assistant generates executable Strudel code from editor context, removing boilerplate so the performer stays in flow while keeping control. It uses Strudel REPL plus Transformers.js with a quantized 1.5B parameters code model and dual runtimes (WebGPU and Wasm). Training uses thousands of validated, LLM‑generated examples with a parse/execute validation gate; switching to direct code generation shrinks the engine by about 90% and delivers low‑latency, offline inference that actually works in a tab.
AI-written summary · about
chapters (15)
- 0:00Live Coding Music in the Browser
- 1:20Discovering Strudel and the Learning Challenge
- 2:52Building AI That Enhances Creativity
- 3:40Demo: An AI Assistant for Strudel
- 5:49Why Run AI Locally in the Browser?
- 7:21Choosing the Right Model and Task
- 9:17The AI–Code Boundary: Models vs Deterministic Logic
- 10:05First Attempts: Prompting and JSON-Based Approaches
- 13:05Simplifying the Architecture with Direct Code Generation
- 15:13Training a Model for a Niche Domain
- 16:33Validate Before You Scale: Generating Training Data
- 18:24Iterating on Data and Improving the Model
- 19:17Fitting a Specialized Model Into a Browser Tab
- 20:50Beyond Music: Patterns for Local AI Applications
- 23:00Final Takeaways & Resources
related talks
36mAs an Artist, AI Is Your Assistant, Not Your Competitor!
20mFrom ML to LLM: On-Device AI in the Browser
33mProgramming as a Performance
12mImprovisation
47mProgramming In Time - Live Coding for Creative Performances
31mFrom ML to LLM on device AI in the browser
08mAIrgots: Live Coding Performance
17mLive Coding Performance