Introducing Visual Scenes and Reasoning: A More Realistic Benchmark for Spoken Language Understanding
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (40) · 33899-33907
DOI: 10.1609/aaai.v40i40.40682 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Di Wu und 9 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).
Abstract (Originalsprache)
Spoken Language Understanding (SLU) consists of two sub-tasks: intent detection (ID) and slot filling (SF). Given its broad range of real-world applications, enhancing SLU for practical deployment is increasingly critical. Profile-based SLU addresses ambiguous user utterances by incorporating context awareness (CA), user profiles (UP), and knowledge graphs (KG) to support disambiguation, thereby advancing SLU research toward real-world applicability. However, existing SLU datasets still fall short in representing real-world scenarios. Specifically, (1) CA uses one-hot vectors for representation, which is overly idealized, and (2) models typically focuses solely on predicting intents and slot labels, neglecting the reasoning process that could enhance performance and interpretability. To overcome these limitations, we introduce VRSLU, a novel SLU dataset that integrates both Visual images and explicit Reasoning. For over-idealized CA, we use GPT-4o and FLUX.1-dev to generate images reflecting users’ environments and statuses, followed by human verification to ensure quality. For reasoning, GPT-4o is employed to generate explanations for predicted labels, which are then refined by human annotators to ensure accuracy and coherence. Additionally, we propose an instructional template, LR-Instruct, which first predicts labels and then generates corresponding reasoning. This two-step approach helps mitigate the influence of reasoning bias on label prediction. Experimental results confirm the effectiveness of incorporating visual information and highlight the promise of explicit reasoning in advancing SLU.
Zitieren
APA
Wu, D., Jiang, L., Fang, R., Bianjing, Xie, H., Su, H., Huang, H., He, Z., Song, S., & Li, X. (2026). Introducing Visual Scenes and Reasoning: A More Realistic Benchmark for Spoken Language Understanding. Proceedings of the AAAI Conference on Artificial Intelligence, 40(40), 33899-33907. https://doi.org/10.1609/aaai.v40i40.40682
Vancouver
Wu D, Jiang L, Fang R, Bianjing, Xie H, Su H, et al. Introducing Visual Scenes and Reasoning: A More Realistic Benchmark for Spoken Language Understanding. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(40):33899-33907. doi:10.1609/aaai.v40i40.40682
BibTeX
@article{Wu2026,
author = {Wu, Di and Jiang, Liting and Fang, Ruiyu and Bianjing and Xie, Hongyan and Su, Haoxiang and Huang, Hao and He, Zhongjiang and Song, Shuangyong and Li, Xuelong},
title = {Introducing Visual Scenes and Reasoning: A More Realistic Benchmark for Spoken Language Understanding},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {40},
pages = {33899--33907},
doi = {10.1609/aaai.v40i40.40682}
}
EUVIMED