Player-Driven Audio Creation in Dreams
summary
Dreams ships a PS4 creation tool that turns motion input from the DualShock 4 into playable instruments: each 2D sound stroke maps to one synth voice and is performed through spatial effect fields that add pitch, volume or time-stretch offsets; gestures record as strokes and play back or convert to piano-roll notes. The audio engine uses fully granular voices, sound gadgets update at ~30 Hz, the animation timeline doubles as a DAW, there’s no mixer UI, and export is currently limited to video capture.
AI-written summary · about
chapters (26)
- 0:00Introduction
- 0:22Dreams Overview
- 1:44Underlying Principles
- 3:08Early Ideas
- 4:23Early Development
- 5:26Music Theory
- 5:59Pitch vs Time
- 6:45Dissonance
- 7:32What a game needs
- 7:55Sound strokes and effect fields
- 8:29Preset interpolator
- 8:52Sound strokes
- 9:40Sound sculpting
- 10:44Dualshock
- 14:18Performance Capture
- 16:09Ranged Parameters
- 18:30View Modes
- 21:02Audio Timeline
- 23:32Unity
- 24:48Audio Engine
- 25:29Music
- 25:55Sound Gadgets
- 26:19Voices
- 26:41Feature Requests
- 27:38Takeaways
- 29:20Future Plans
related talks
01h 10mThe Future of Audio in Unreal Engine
41mLet the player be the performer
29mIntroduction to game audio
54mMerging Interactive Narrative with Live Action
23mFireside Chat: The next-gen console opportunity
42mExploring the sonic possibilities in sci-fi with Dome Keeper
17mHow to create FPS using audio not graphics
27mPython and TouchDesigner for Interactive Experiments