Keynote 3. Interactive generative models for music
summary
An open-source Magenta stack turns generative models into interactive tools: a Coconet-backed Bach Doodle harmonizes 55 million melodies in two days, while a browser-side model moves from a 60-layer conv that takes ~40 seconds for two bars to ~2 seconds via dilated convolutions and op fusion (TensorFlow.js). Interaction uses an infilling API with Gibbs-style rewriting for iterative edits. Music Transformer adds relative attention and 10 ms event timing to model expressive disklavier performances. An AI Song Contest winner chains melody priming (Daisy), audio conditioning and singer-conditioned synthesis into a feedback loop.
AI-written summary · about
chapters (15)
- 0:00Intro
- 0:30Prelude
- 3:44Agenda
- 5:00Motivation
- 12:40Infilling model
- 18:25Speed up
- 23:42Models
- 29:02Attention
- 33:17Creative process
- 35:05Demo
- 39:32Question
- 49:24Lab in the Wild
- 50:57Challenges
- 52:40Winning entry
- 58:07Conclusion
related talks
36mMusic and Art with ML
41mMusic and Machine Learning
28mMusic and AI in the Browser with TensorFlow js and Magenta js
21mMusical Deep Neural Networks in the Browser
24mCreating an AI Musician with JavaScript
46mMusic Generation with Magenta: Using Machine Learning in
32mGetting started with generative AI with AWS DeepComposer
05mInteractive Exploration-Exploitation Balancing for Generative Melody Composition