Anchoring Bias Affects Mental Models and User Reliance in Explainable AI Systems
summary
Controlled user study builds an explainable video activity recognition tool for a cooking domain and tests 8 policies (4 strengths and 4 weaknesses) with order and explanation presence manipulated. Main claim is that positive-first users make significantly more errors because they over-rely on the model while using the system more, yet they form a more accurate estimate of specific competencies; negative-first users under-rely, make fewer errors, and underestimate model ability. Explanations alone do not eliminate the anchoring effect and overconfidence.
AI-written summary · about
related talks
46mAI for High-Stakes Decision Making
19mMoving Forward with Explainable AI: Adoption Rate & Essential Dimensions for a Reliable Use
37mInteractive Explainable AI
46mCan you trust your AI?
50mSecurity as Part of Responsible AI: At Home or At Odds?
39mKeynote 3: How did you know? Explaining Black Box Model Predictions in Python
39mThe Search for Transparency and Accountability in the Age of...
39mGuide towards algorithm explainability in machine learning