Removing Soft Shadows with Hard Data
summary
Claims a pipeline that learns soft-shadow mattes from photorealistic rendering and uses them to remove or edit shadows in RGB photos with only a brush mask. It trains a random forest nearest-neighbor regressor on tens of thousands of rendered pairs, uses guided inpainting as an initial guess, patch alignment, MRF regularization (TRW-S), and a per-channel color scaling grid search to recover color. Evaluation includes a 50-participant user study showing perceptual improvement over prior methods; works well for soft shadows but fails when inpainting utterly breaks.
AI-written summary · about
chapters (14)
- 0:00Shadow removal research
- 1:24User-provided mask shadow removal
- 2:48Soft shadows explained
- 5:39Penumbra from synthetic data
- 7:03Content-aware fill initial guess
- 9:59Patch alignment for features
- 11:46Random forest nearest neighbor
- 13:55Feature extraction importance
- 16:04Regularization with message passing
- 17:30Recovering shadow colors
- 19:39Pipeline and results
- 21:53User study evaluation
- 23:15Limitations and future
- 24:20Conclusion
related talks
28mBackground removal without background knowledge
38mShadows Generation in the Wild
06m#ProjectDeepFill
14mRecreational Image Reconstruction with Decision Trees
59mStudying Failure Modes of Deep Learning and Ways To Fix Them
27mAI for social good
01h 00mCloud and Shadow Recognition in Satellite Imagery
40mLearn to be a painter using Neural Style Painting