EUVIMEDEUVIMED entdecken
Crossref

Introducing Visual Scenes and Reasoning: A More Realistic Benchmark for Spoken Language Understanding

Di Wu, Liting Jiang, Ruiyu Fang, Bianjing, Hongyan Xie, Haoxiang Su, Hao Huang, Zhongjiang He, Shuangyong Song, Xuelong Li

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (40) · 33899-33907

DOI: 10.1609/aaai.v40i40.40682 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Di Wu und 9 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Spoken Language Understanding (SLU) consists of two sub-tasks: intent detection (ID) and slot filling (SF). Given its broad range of real-world applications, enhancing SLU for practical deployment is increasingly critical. Profile-based SLU addresses ambiguous user utterances by incorporating context awareness (CA), user profiles (UP), and knowledge graphs (KG) to support disambiguation, thereby advancing SLU research toward real-world applicability. However, existing SLU datasets still fall short in representing real-world scenarios. Specifically, (1) CA uses one-hot vectors for representation, which is overly idealized, and (2) models typically focuses solely on predicting intents and slot labels, neglecting the reasoning process that could enhance performance and interpretability. To overcome these limitations, we introduce VRSLU, a novel SLU dataset that integrates both Visual images and explicit Reasoning. For over-idealized CA, we use GPT-4o and FLUX.1-dev to generate images reflecting users’ environments and statuses, followed by human verification to ensure quality. For reasoning, GPT-4o is employed to generate explanations for predicted labels, which are then refined by human annotators to ensure accuracy and coherence. Additionally, we propose an instructional template, LR-Instruct, which first predicts labels and then generates corresponding reasoning. This two-step approach helps mitigate the influence of reasoning bias on label prediction. Experimental results confirm the effectiveness of incorporating visual information and highlight the promise of explicit reasoning in advancing SLU.

Zitieren

APA

Wu, D., Jiang, L., Fang, R., Bianjing, Xie, H., Su, H., Huang, H., He, Z., Song, S., & Li, X. (2026). Introducing Visual Scenes and Reasoning: A More Realistic Benchmark for Spoken Language Understanding. Proceedings of the AAAI Conference on Artificial Intelligence, 40(40), 33899-33907. https://doi.org/10.1609/aaai.v40i40.40682

Vancouver

Wu D, Jiang L, Fang R, Bianjing, Xie H, Su H, et al. Introducing Visual Scenes and Reasoning: A More Realistic Benchmark for Spoken Language Understanding. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(40):33899-33907. doi:10.1609/aaai.v40i40.40682

BibTeX
@article{Wu2026,
  author = {Wu, Di and Jiang, Liting and Fang, Ruiyu and Bianjing and Xie, Hongyan and Su, Haoxiang and Huang, Hao and He, Zhongjiang and Song, Shuangyong and Li, Xuelong},
  title = {Introducing Visual Scenes and Reasoning: A More Realistic Benchmark for Spoken Language Understanding},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {40},
  pages = {33899--33907},
  doi = {10.1609/aaai.v40i40.40682}
}
Kostenlos starten →