Enhancing Accessibility with AI and ML
summary
Deque uses computer vision and OCR plus human-verified labels to infer UI semantics and surface mismatches between visual roles and markup, claiming it boosts automatic detection of accessibility issues. Models train on hundreds of thousands of UI screenshots with expert labelers, use model-assisted pre-annotation, and deploy to products like the Interactive Elements and Table tools and an axe for designers Figma plugin. A continuous pipeline yields biweekly model updates, two-week fix cycles, and low miss rates on common controls.
AI-written summary · about
chapters (19)
- 0:00Introduction to ml in accessibility
- 0:33Evolution of machine learning
- 2:41General AI vs specialized models
- 4:15ML in accessibility testing
- 5:28Accounting for machine learning errors
- 6:38Computer vision for UI analysis
- 8:18Visual examples of UI analysis
- 9:18Interactive elements tool
- 11:44Complex page structure detection
- 14:17Guided fixes and UI relationships
- 19:12Table structure interpretation
- 21:59Technical challenges in UI interpretation
- 24:22Object detection network design
- 26:08Impact of ML on testing workflow
- 27:27DQ systems data pipeline
- 30:49Continuous learning workflow
- 33:42Summary of accessibility benefits
- 34:23Acts for designers plugin
- 37:32Q&A session
related talks
47mAI Systems for Accessibility
50mHuman-Centered AI and Accessibility
19mAgile Accessible Software Development
34mAdding accessibility into Unit Tests
17mNew options for accessible design
49mUsing AI to Streamline Inclusive Design Without Losing Humanity
49mHow Do You Make Accessibility Testing as Efficient as Possible
28mAutomated Accessibility Testing