An Assessment of Human–AI Interaction Capability in the Generative AI Era: The Influence of Critical Thinking
Journal of Intelligence · 26.05.2025 · 13 (6) · 62
DOI: 10.3390/jintelligence13060062 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Intelligence, online erschienen am 26.05.2025. Verfasst von Feiming Li und 4 weiteren Autor:innen. Beteiligte Einrichtung: Zhejiang Key Laboratory of Intelligent Education Technology and Application, Zhejiang Normal University, Jinhua 321004, China. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: social sciences department of the ministry of education, Zhejiang Provincial Philosophy and Social Sciences Planning Project, Humanities and Social Sciences Research Project of Ministry of Education, China.
Abstract (Originalsprache)
(1)
Background
In the era of generative AI (GenAI), assessing AI literacy is essential for understanding how effectively non-expert users can interact with AI. However, existing assessment tools primarily focus on users’ understanding of AI principles or rely on self-reported scales, neglecting critical thinking and actual interaction capabilities. To address this gap, this study aims to design and validate evaluation indicators targeting the behavioral process of human–GenAI interactions and analyze the impact of critical thinking. (2)
Methods
Grounded in information literacy and critical thinking frameworks, this study operationalized human–AI interaction capabilities into behavioral indicators and rubrics through observation, surveys, and pilot studies. Data were collected from 121 undergraduates completing two real-world tasks with GenAI, and their interaction processes were documented and evaluated. (3)
Results
The indicators showed acceptable inter-rater and internal consistency reliability. Exploratory and Confirmatory Factor Analysis confirmed a three-dimensional structure. Further analysis showed that interaction capabilities varied across gender, academic background, AIGC use frequency, critical thinking disposition levels, and question chain logic. (4)
Conclusions
The developed evaluation indicators are reliable and valid. Further analysis reveals that a high critical thinking disposition can offset the disadvantage of lower usage frequency. This highlights the significance of critical thinking in enhancing human–GenAI interaction capabilities.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Zhejiang Key Laboratory of Intelligent Education Technology and Application, Zhejiang Normal University, Jinhua 321004, China.
- College of Education, Zhejiang Normal University, Jinhua 321004, China.
- School of Psychology, Zhejiang Normal University, Jinhua 321004, China.
Förderung
- social sciences department of the ministry of education
- Zhejiang Provincial Philosophy and Social Sciences Planning Project
- Humanities and Social Sciences Research Project of Ministry of Education, China
Zitieren
APA
Li, F., Yan, X., Su, H., Shen, R., & Mao, G. (2025). An Assessment of Human–AI Interaction Capability in the Generative AI Era: The Influence of Critical Thinking. Journal of Intelligence, 13(6), 62. https://doi.org/10.3390/jintelligence13060062
Vancouver
Li F, Yan X, Su H, Shen R, Mao G. An Assessment of Human–AI Interaction Capability in the Generative AI Era: The Influence of Critical Thinking. Journal of Intelligence. 2025;13(6):62. doi:10.3390/jintelligence13060062
BibTeX
@article{Li2025,
author = {Li, Feiming and Yan, Xinyu and Su, Hongli and Shen, Rong and Mao, Gang},
title = {An Assessment of Human–AI Interaction Capability in the Generative AI Era: The Influence of Critical Thinking},
journal = {Journal of Intelligence},
year = {2025},
volume = {13},
number = {6},
pages = {62},
doi = {10.3390/jintelligence13060062}
}
EUVIMED