~/talk

NoteIt: A System Converting Instructional Videos to Interactable Notes Through Multimodal Video U...

on YouTube 11m
Talkat UIST 2025 · 2025-09-28..10-01· [developer-experience]

speakers

name:Run Zhao
links:none on file
name:Zhihan Jiang
links:none on file
name:Xinchen Zhang
links:none on file
name:Chirui Chang
links:none on file
name:Handi Chen
links:none on file
name:Weipeng Deng
links:none on file
name:Luyao Jin
links:none on file
name:xiaojuan qi
links:none on file
name:Xun Qian
links:none on file
name:Edith C. H. Ngai
links:none on file

resources

video:https://youtu.be/JIg4aGuJ6r4
slides:none on file

summary

NoteIt automatically converts instructional videos into editable, interactable notes using a pipeline that extracts hierarchical structure extraction (chapters and steps via video-differential captions + GPT-4 clustering and chain-of-thought DAG), captures multimodal key content (Whisper transcripts, static and dynamic key frames, GPT-4 vision planner/executor), and maps everything into node schemas. Technical evaluation uses 32 videos and reports high recall for static frames and high precision for dynamic frames; a within-subject user study (N=36) rates NoteIt higher than a commercial baseline on consistency, informativeness, adaptability, and satisfaction.

AI-written summary · about

chapters (7)

  1. 0:00Introduction and motivation
  2. 1:29Design space analysis
  3. 3:39System design and pipeline
  4. 6:30Demo and user interaction
  5. 7:55Evaluation results
  6. 8:38Insights and future work
  7. 10:04Q&A and comparison

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.