“Are you sure about that?” The effects of calibrated classification model task accuracy and confidence on trustworthiness, trust, and performance
Applied Ergonomics · 2026
DOI: 10.1016/j.apergo.2025.104657 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Ergonomics, 2026. Verfasst von August Capiola und 6 weiteren Autor:innen. Beteiligte Einrichtung: Air Force Research Laboratory, 2215 First St., WPAFB, OH, 45433, USA. Electronic address: august.capiola.1@us.af.mil. Hauptthemen laut MEDLINE: Trust, Task Performance and Analysis, Machine Learning, Artificial Intelligence. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 42 Literaturangaben. Gefördert von: Air Force Office of Scientific Research, Air Force Research Laboratory.
Abstract (Originalsprache)
Artificial intelligence and machine learning are becoming increasingly popular. With their deployment comes the challenge of ensuring people have appropriate expectations of their function to ensure calibrated reliance. As these models are traditionally opaque, emerging models are attempting to better convey their confidence and accuracy to users. The present research expands on recent work, investigating the effects of models' classification confidence and accuracy on trust-relevant criteria. In a within-subjects design, participants leveraged models of varying confidence and accuracy in an online image classification task. Results demonstrated the effects of model classification accuracy were qualified by its classification confidence. Post hoc analyses showed that in most cases, models conveying high confidence and low accuracy were perceived as less trustworthy, trusted less, and resulted in lower task performance and increased decision time. The results replicate previous work and expand on the implications that model confidence and accuracy have on trust-relevant criteria.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Air Force Research Laboratory, 2215 First St., WPAFB, OH, 45433, USA. Electronic address: august.capiola.1@us.af.mil.
- DCS Corporation, 4027 Col. Glenn Hwy., Beavercreek, OH, 45431, USA.
- Air Force Research Laboratory, 2215 First St., WPAFB, OH, 45433, USA.
- Air Force Research Laboratory, 525 Brooks Rd., Rome, NY, 13441, USA.
Förderung
- Air Force Office of Scientific Research
- Air Force Research Laboratory
Zitieren
APA
Capiola, A., Harris, K. N., Alarcon, G. M., Johnson, D., Jessup, S. A., Willis, S. M., & Bennette, W. (2026). “Are you sure about that?” The effects of calibrated classification model task accuracy and confidence on trustworthiness, trust, and performance. Applied Ergonomics. https://doi.org/10.1016/j.apergo.2025.104657
Vancouver
Capiola A, Harris KN, Alarcon GM, Johnson D, Jessup SA, Willis SM, et al. “Are you sure about that?” The effects of calibrated classification model task accuracy and confidence on trustworthiness, trust, and performance. Applied Ergonomics. 2026. doi:10.1016/j.apergo.2025.104657
BibTeX
@article{Capiola2026,
author = {Capiola, August and Harris, Krista N and Alarcon, Gene M and Johnson, Dexter and Jessup, Sarah A and Willis, Sasha M and Bennette, Walter},
title = {“Are you sure about that?” The effects of calibrated classification model task accuracy and confidence on trustworthiness, trust, and performance},
journal = {Applied Ergonomics},
year = {2026},
doi = {10.1016/j.apergo.2025.104657}
}
EUVIMED