Teaching WebGL to Dance to Music

on YouTube ↗36m

" summary

A prototype moves from SVG animation that pegged one CPU at ~40% to WebGL via the Elm package elm-3d-scene, adding true 3D layers, rotations and colors. Tempo comes from WebMIDI for exact note timing instead of audio beat detection. A small DSL called dance lane maps MIDI notes to body part translations and rotations. The final demo is a 3D dancer that responds to live MIDI in real time and performs reliably on recorded runs.

AI-written summary · about

" chapters (16)

1. 0:00Introduction
2. 1:12Goals and background
3. 2:58Dance choreography methods
4. 7:17Encoding dance in SVG
5. 10:48Why WebGL for 3D
6. 13:20WebGL basics and Shader Toy
7. 16:08Elm 3D Scene for 3D
8. 18:34Syncing animation with music
9. 21:27Dance Lane DSL
10. 23:11Final demo
11. 24:57Talk recap and shoutouts
12. 26:08Q&A: Elm
13. 27:53Q&A: SVG vs WebGL
14. 29:01Q&A: Elm 3D Scene
15. 31:30Q&A: OP-1 music
16. 33:35Q&A: Elm packages

" related talks

• 24mBuilding a Music Learning Game with Elm, Web MIDI, and SVG AnimationLiz Krane · ReactiveConf 2019
• 26m"Building a Music Learning Game with Elm, Web MIDI, and SVG Animation"Liz Krane · Elm Conf 2019
• 30mJavaScript Patterns For Contemporary Dance MusicJan Krutisch · JSConf EU 2013
• 20mLook What You MIDI Me Do!Rachel White · JSConf Hawaiʻi 2020
• 14mMusical Creativity in the BrowserScott Ammon · CascadiaJS 2020
• 31mMakepad - Rethinking Web UI's with WebGLRik Arends · AmsterdamJS Conference 2017
• 40mLiveJS: Making Music with JavaScriptRuth John, Martin Schuhfuss, Matt Mckegg · ColdFront 2017
• 34mBringing the Fun to Graphics ProgrammingAndrey Kuzmin · Lambda Days 2018