Robust Annotation of Mobile Application Interfaces in Methods for Accessibility Repair and ...
summary
Develops methods to attach reusable accessibility metadata to mobile UI elements using screen identifiers, element identifiers, and screen equivalence heuristics; implements a web annotation interface and runtime overlays (interaction proxies) that add alt text and clickable regions. Evaluates on 2,038 screens from 50 apps, reduces FalseSame errors by 97% and FalseDifferent by 60%, demonstrates three runtime repairs, fixes ~40 issues for one long‑time TalkBack user and repairs issues across 26 real apps.
AI-written summary · about
chapters (22)
- 0:00Intro
- 0:38Mobile Accessibility Matters
- 0:57Not All Mobile Apps are Accessible
- 1:56Repair Accessibility Issue
- 2:22Repair One Accessibility Issue
- 3:11Mobile App UI Annotation for Repair
- 3:28Talk Outline
- 3:57Scalable Web Social Annotation
- 5:47Identifiers for Mobile UI Annotation
- 6:02Identifiers for UI Annotation
- 6:37Screen Identification
- 8:28Approaches to Screen Equivalence
- 9:57Our Approach to Screen Equivalence
- 10:47Screen Equivalence Heuristic Evaluation
- 14:10Element identification
- 14:24Create Annotation for Different Issues
- 14:53Evaluation of Runtime Repair
- 15:14Repair upon Request
- 15:52Repair at Larger Scale
- 16:19Future Opportunities
- 17:01Summary of Contributions
- 17:36Robust Annotation of Mobile Application Interfaces in Methods for Accessibility Repair and Enhancement
related talks
46mMobile accessibility - challenges and best practices
42mMake Your Mobile Apps Accessible to All
35mAccessibility in Action
39mHow to win at mobile accessibility
37mAndroid accessibility made easy
24mBenefits of Making Your Apps Accessible
40mAccessibility on Android in 2019
37mAccessibility on Android in 2019