Objective Measures of Cognitive Load Using Deep Multi-Modal Learning — A Use-Case in Aviation
summary
Using a mixed VR and physical simulator plus C-17 flight data, researchers train a BM3TX architecture (variational auto-encoders per sensor, x-vector–style stat pooling) on 40 subjects (21 pilots, 9 operators, 10 novices) with HTC Vive Pro Eye and E4 Empatica sensors measuring pupillometry, EDA, PPG, temperature and throttle-hand acceleration. Cross-subject binary classification exceeds 80% accuracy in simulation, multi-task training is required, throttle acceleration and EDA are most informative, and a pretrained model achieves a 90% true positive rate on five real flights.
AI-written summary · about
related talks
07mEnhancing In-game Immersion Using BCI-controlled Mechanics
12mThe Potential of Cognitive Circles to Measure Mental Load
25mInvestigating the Opportunities for Utilizing AI and XR Technologies in the Future of Training
12mUnderstanding Driving Distractions: A Multimodal Analysis on Distraction Characterization
01h 23mMultimodal Deep Learning in the Real World
19mNeuro Game Experience: Your Thought Takes Control!
45mAI in Selection of Candidates
18mAdding Human Learning in Brain--Computer Interfaces (BCIs): Towards a Practical Control Modality