Photogrammetry workflow and the tech behind the de-lighting tool
summary
Unity describes a full photogrammetry pipeline from $3k–$8k capture rigs through DCraw calibration, RealityCapture alignment and mesh creation, Instant Meshes retopology, xNormal/Substance baking and PLY exports to produce game-ready assets used in the "Font and Blow" demo. The open‑source de-lighting tool removes baked illumination by estimating a directional irradiance map from the surface using a color checker, runs on 4K textures instantly, largely succeeds but leaves strong cast shadows (experimental repair mask in progress). RealityCapture (≈10× speedup) is their preferred reconstructor.
AI-written summary · about
chapters (39)
- 0:00Introduction
- 0:38Context
- 1:53Demo
- 5:26Photogrammetry game
- 8:31Photogrammetry tips
- 10:45Camera parameters
- 13:10Video capture
- 14:15Planning
- 14:36References
- 14:56Shooting objects
- 15:31Ground
- 16:48Wall
- 17:08Details
- 18:17Processing
- 18:42Software
- 18:56Hardware
- 19:11Calibration
- 19:44Aligning
- 20:16Components
- 20:36Control Points
- 20:50Modeling
- 21:39Simplifying
- 21:56Exporting
- 22:27Meshing
- 22:39Texture Baking
- 23:11Software used
- 23:41Textures
- 28:43Environment Lighting
- 29:27Surface Color
- 33:41Rough Texture
- 34:37Directional Albedo
- 36:32Not spherical Albedo
- 37:16Global and local lighting
- 38:16Prop surfaces
- 39:21Ambient occlusion
- 41:55Repair mask
- 42:47Custom textures
- 44:50Conclusion
- 45:18Questions
related talks
30mCapturing real-world materials for real-time development
17mOptimizing Portrait Lighting at Capture-Time Using a 360 Camera as a Light Probe
17mData Challenges with 3D Computer Vision #bbuzz
09mOctaneRender Updates
36mPushing the MetaHuman Likeness Limits via Scan-based Textures
15mDepthLab: Real-time 3D Interaction with Depth Maps for Mobile Augmented Reality
33mNASA Lunar Terrain Visualization in Unreal Engine 5
47mThe Visual Technology of Gears 5