EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

Assessing Interaction Quality in Human–AI Dialogue: An Integrative Review and Multi-Layer Framework for Conversational Agents

Luca Marconi, Luca Longo, Federico Cabitza

Machine Learning and Knowledge Extraction · 2026-01-26 · 8 (2) · 28

DOI: 10.3390/make8020028 ↗

At a glance

Journal article in Machine Learning and Knowledge Extraction, published online on 2026-01-26. Written by Luca Marconi and 2 co-authors. Cited 16 times so far (Crossref); 107 references. The full text is freely available (open access, CC BY 4.0).

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Conversational agents are transforming digital interactions across various domains, including healthcare, education, and customer service, thanks to advances in large language models (LLMs). As these systems become more autonomous and ubiquitous, understanding what constitutes high-quality interaction from a user perspective is increasingly critical. Despite growing empirical research, the field lacks a unified framework for defining, measuring, and designing user-perceived interaction quality in human–artificial intelligence (AI) dialogue. Here, we present an integrative review of 125 empirical studies published between 2017 and 2025, spanning text-, voice-, and LLM-powered systems. Our synthesis identifies three consistent layers of user judgment: a pragmatic core (usability, task effectiveness, and conversational competence), a social–affective layer (social presence, warmth, and synchronicity), and an accountability and inclusion layer (transparency, accessibility, and fairness). These insights are formalised into a four-layer interpretive framework—Capacity, Alignment, Levers, and Outcomes—operationalised via a Capacity × Alignment matrix that maps distinct success and failure regimes. It also identifies design levers such as anthropomorphism, role framing, and onboarding strategies. The framework consolidates constructs, positions inclusion and accountability as central to quality, and offers actionable guidance for evaluation and design. This research redefines interaction quality as a dialogic construct, shifting the focus from system performance to co-orchestrated, user-centred dialogue quality.

Cite

APA

Marconi, L., Longo, L., & Cabitza, F. (2026). Assessing Interaction Quality in Human–AI Dialogue: An Integrative Review and Multi-Layer Framework for Conversational Agents. Machine Learning and Knowledge Extraction, 8(2), 28. https://doi.org/10.3390/make8020028

Vancouver

Marconi L, Longo L, Cabitza F. Assessing Interaction Quality in Human–AI Dialogue: An Integrative Review and Multi-Layer Framework for Conversational Agents. Machine Learning and Knowledge Extraction. 2026;8(2):28. doi:10.3390/make8020028

BibTeX
@article{Marconi2026,
  author = {Marconi, Luca and Longo, Luca and Cabitza, Federico},
  title = {Assessing Interaction Quality in Human–AI Dialogue: An Integrative Review and Multi-Layer Framework for Conversational Agents},
  journal = {Machine Learning and Knowledge Extraction},
  year = {2026},
  volume = {8},
  number = {2},
  pages = {28},
  doi = {10.3390/make8020028}
}
Start for free →