EUVIMEDEUVIMED entdecken
Crossref

“Are you sure about that?” The effects of calibrated classification model task accuracy and confidence on trustworthiness, trust, and performance

August Capiola, Krista N Harris, Gene M Alarcon, Dexter Johnson, Sarah A Jessup, Sasha M Willis, Walter Bennette

Applied Ergonomics · 2026

DOI: 10.1016/j.apergo.2025.104657 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Ergonomics, 2026. Verfasst von August Capiola und 6 weiteren Autor:innen. Beteiligte Einrichtung: Air Force Research Laboratory, 2215 First St., WPAFB, OH, 45433, USA. Electronic address: august.capiola.1@us.af.mil. Hauptthemen laut MEDLINE: Trust, Task Performance and Analysis, Machine Learning, Artificial Intelligence. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 42 Literaturangaben. Gefördert von: Air Force Office of Scientific Research, Air Force Research Laboratory.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Artificial intelligence and machine learning are becoming increasingly popular. With their deployment comes the challenge of ensuring people have appropriate expectations of their function to ensure calibrated reliance. As these models are traditionally opaque, emerging models are attempting to better convey their confidence and accuracy to users. The present research expands on recent work, investigating the effects of models' classification confidence and accuracy on trust-relevant criteria. In a within-subjects design, participants leveraged models of varying confidence and accuracy in an online image classification task. Results demonstrated the effects of model classification accuracy were qualified by its classification confidence. Post hoc analyses showed that in most cases, models conveying high confidence and low accuracy were perceived as less trustworthy, trusted less, and resulted in lower task performance and increased decision time. The results replicate previous work and expand on the implications that model confidence and accuracy have on trust-relevant criteria.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

TrustTask Performance and AnalysisMachine LearningArtificial IntelligenceHumansMaleFemaleYoung AdultAdultCalibrationDecision Making

Autor:innen und Institutionen

  • Air Force Research Laboratory, 2215 First St., WPAFB, OH, 45433, USA. Electronic address: august.capiola.1@us.af.mil.
  • DCS Corporation, 4027 Col. Glenn Hwy., Beavercreek, OH, 45431, USA.
  • Air Force Research Laboratory, 2215 First St., WPAFB, OH, 45433, USA.
  • Air Force Research Laboratory, 525 Brooks Rd., Rome, NY, 13441, USA.

Förderung

  • Air Force Office of Scientific Research
  • Air Force Research Laboratory

Zitieren

APA

Capiola, A., Harris, K. N., Alarcon, G. M., Johnson, D., Jessup, S. A., Willis, S. M., & Bennette, W. (2026). “Are you sure about that?” The effects of calibrated classification model task accuracy and confidence on trustworthiness, trust, and performance. Applied Ergonomics. https://doi.org/10.1016/j.apergo.2025.104657

Vancouver

Capiola A, Harris KN, Alarcon GM, Johnson D, Jessup SA, Willis SM, et al. “Are you sure about that?” The effects of calibrated classification model task accuracy and confidence on trustworthiness, trust, and performance. Applied Ergonomics. 2026. doi:10.1016/j.apergo.2025.104657

BibTeX
@article{Capiola2026,
  author = {Capiola, August and Harris, Krista N and Alarcon, Gene M and Johnson, Dexter and Jessup, Sarah A and Willis, Sasha M and Bennette, Walter},
  title = {“Are you sure about that?” The effects of calibrated classification model task accuracy and confidence on trustworthiness, trust, and performance},
  journal = {Applied Ergonomics},
  year = {2026},
  doi = {10.1016/j.apergo.2025.104657}
}
Kostenlos starten →