Neuro-symbolic procedural semantics for explainable visual dialogue
PLOS One · 27.05.2025 · 20 (5) · e0323098
DOI: 10.1371/journal.pone.0323098 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS One, online erschienen am 27.05.2025. Verfasst von Lara Verheyen und 4 weiteren Autor:innen. Beteiligte Einrichtung: Artificial Intelligence Lab, Vrije Universiteit Brussel, Brussels, Belgium. Hauptthemen laut MEDLINE: Semantics, Communication. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 86 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: HORIZON EUROPE European Innovation Council, Vlaamse Overheid, Research Foundation - Flanders.
Abstract (Originalsprache)
This paper introduces a novel approach to visual dialogue that is based on neuro-symbolic procedural semantics. The approach builds further on earlier work on procedural semantics for visual question answering and expands it with neuro-symbolic mechanisms that handle the challenges that are inherent to dialogue, in particular the incremental nature of the information that is conveyed. Concretely, we introduce (i) the use of a conversation memory as a data structure that explicitly and incrementally represents the information that is expressed during the subsequent turns of a dialogue, and (ii) the design of a neuro-symbolic procedural semantic representation that is grounded in both visual input and the conversation memory. We validate the methodology using the MNIST Dialog and CLEVR-Dialog benchmark challenges and achieve a question-level accuracy of 99.8% and 99.2% respectively. The methodology presented in this paper contributes to the growing body of research in artificial intelligence that tackles tasks that involve both low-level perception and high-level reasoning using a combination of neural and symbolic techniques. It thereby leads the way towards the development of conversational agents that will be able to hold more explainable, natural and coherent conversations with their human interlocutors.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Artificial Intelligence Lab, Vrije Universiteit Brussel, Brussels, Belgium.
- Faculté d'informatique, Université de Namur, Namur, Belgium.
ORCID-Profile: Paul Van Eecke, Katrien Beuls
Förderung
- HORIZON EUROPE European Innovation Council
- Vlaamse Overheid
- Research Foundation - Flanders
- Gouvernement Wallon
Zitieren
APA
Verheyen, L., Botoko Ekila, J., Nevens, J., Van Eecke, P., & Beuls, K. (2025). Neuro-symbolic procedural semantics for explainable visual dialogue. PLOS One, 20(5), e0323098. https://doi.org/10.1371/journal.pone.0323098
Vancouver
Verheyen L, Botoko Ekila J, Nevens J, Van Eecke P, Beuls K. Neuro-symbolic procedural semantics for explainable visual dialogue. PLOS One. 2025;20(5):e0323098. doi:10.1371/journal.pone.0323098
BibTeX
@article{Verheyen2025,
author = {Verheyen, Lara and Botoko Ekila, Jérôme and Nevens, Jens and Van Eecke, Paul and Beuls, Katrien},
title = {Neuro-symbolic procedural semantics for explainable visual dialogue},
journal = {PLOS One},
year = {2025},
volume = {20},
number = {5},
pages = {e0323098},
doi = {10.1371/journal.pone.0323098}
}
EUVIMED