Adaptive Wavelet Distillation from DNNs and Dictionary Learning
summary
Pretrained ResNet outputs are distilled into an adaptive wavelet transform with a three-term loss (reconstruction + wavelet constraints + an L1 gradient interpretation penalty), producing as few as 30 coefficients that shave about 10% off ResNet errors on simulated cosmology Omega_m estimation and raise R^2 from 0.23 to 0.26 on a clathrin endocytosis time-series prediction. Unsupervised work extracts 21 principal patterns from Drosophila gene-expression images (reconstructing the gap-gene network except one link) and proves near–necessary/sufficient identifiability conditions for Bernoulli–Gaussian and nonnegative models.
AI-written summary · about
chapters (13)
- 0:00Intro
- 0:35Scientific Machine Learning
- 2:02Interpretation
- 2:29Interpretation Learning
- 3:42Relevancy
- 4:27Results
- 11:01Model distillation
- 21:35Peak counting
- 26:29Unsupervised Extreme Learning
- 27:04Development Biology
- 55:45Current Directions
- 56:43Local Convergence
- 57:36Interpretability
related talks
47mInterpreting Deep Neural Networks (DNNs)
32mFinetuning Stable Diffusion with low-rank adaptation techniques
01h 07mDeep Learning with TensorFlow
01h 04mGeneralization in Diffusion Models Arises from Geometry-Adaptive Harmonic Rrepresentations
45mSolving Inverse Problems with Deep Learning
27mTurning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring
26mHit-reco: ProtoDUNE Raw Data Denoising with DL models
59mStudying Failure Modes of Deep Learning and Ways To Fix Them