Where are we looking? Predicting human gaze using deep networks.
summary
Claims a low‑level intensity contrast model matches or beats many classic saliency methods and that a learned Gabor bank narrows the gap to high‑level models. Uses VGG-19 conv5 features (≈2,500 maps) with a four‑layer conv readout, softmaxed into a fixation probability map. Trains in TensorFlow, pretrains on SALICON mouse‑tracking then fine‑tunes on ~1,000 eye‑tracking images. Reports an upper bound ~2.1 bits/fixation, center‑bias baseline ~0.9 bits, and top models explaining only ~30–33% of the gold standard, so deep features help but low‑level cues remain surprisingly powerful.
AI-written summary · about
chapters (11)
- 0:00Human gaze prediction intro
- 2:08Why study gaze matters
- 6:23Visual features driving attention
- 9:17Modeling saliency maps
- 16:03Proposed saliency architecture
- 21:02Eye tracking data challenges
- 23:50Training pipeline and metrics
- 26:25Model predictions comparison
- 29:35Low and high-level features
- 30:36Resources and tools
- 31:19Q&A
related talks
42mLearning Representations for Visual Search
19mWhat Can Be Predicted from Six Seconds of Driver Glances?
48mEye Tracking (a Deep Learning Approach)
10mHAGI: Head-Assisted Gaze Imputation for Mobile Eye Trackers
09mQuantitative Evaluation of Machine Learning Explanations: A Human-Grounded Benchmark
27mUnderstanding Deep Neural Networks
20mLearning Visual Importance for Graphic Designs and Data Visualizations
32mHeart to Heart and Eye to Eye with Deep Fakes