EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

M 3 QuestionIng : Multi-modal Multi-span Medical Question Answering

Anisha Saha, Vaibhav Rathore, Abhisek Tiwari, Akash Ghosh, Sai Ruthvik Edara, Sriparna Saha

ACM Transactions on Computing for Healthcare · 2026-10-31 · 7 (4) · 1-19

DOI: 10.1145/3820162 ↗

At a glance

Journal article in ACM Transactions on Computing for Healthcare, published online on 2026-08-19. Written by Anisha Saha and 5 co-authors. 71 references. The full text is freely available (open access, CC BY 4.0).

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

The growing adoption of AI in healthcare, particularly in preventive care, highlights the critical need for accessibility and precision in Medical Question Answering (MedQA). In recent years, significant efforts have been made to develop multi-span MedQA systems, where the answer to a query may span multiple sections or paragraphs of a source document. However, existing systems fall short of aligning with real-world scenarios, where source documents often include both textual and visual content, requiring answers to incorporate images for better comprehension. To address this gap, we propose \(M^{3}QAFrame\) , a multi-modal, multi-span MedQA framework that leverages visual cues to enhance the generation of comprehensive answers drawn from diverse textual and visual spans. The model takes the context, query, and images as input and outputs an answer containing both textual spans and relevant images. The text and image embeddings are processed using a transformer-based architecture to determine the sentence and image relevance. Additionally, we curate a multi-modal, multi-span MedQA dataset, \(M^{3}QuestionIng\) , containing queries, medical contexts, associated medical images, and extractive answers. Each query–answer pair is labeled with user intent and query type to enhance query and context comprehension. Extensive experiments show that our approach consistently outperforms existing methods across various evaluation metrics. Code and dataset are available at https://github.com/anisha0325/M3QA .

Cite

APA

Saha, A., Rathore, V., Tiwari, A., Ghosh, A., Edara, S. R., & Saha, S. (2026). M 3 QuestionIng : Multi-modal Multi-span Medical Question Answering. ACM Transactions on Computing for Healthcare, 7(4), 1-19. https://doi.org/10.1145/3820162

Vancouver

Saha A, Rathore V, Tiwari A, Ghosh A, Edara SR, Saha S. M 3 QuestionIng : Multi-modal Multi-span Medical Question Answering. ACM Transactions on Computing for Healthcare. 2026;7(4):1-19. doi:10.1145/3820162

BibTeX
@article{Saha2026,
  author = {Saha, Anisha and Rathore, Vaibhav and Tiwari, Abhisek and Ghosh, Akash and Edara, Sai Ruthvik and Saha, Sriparna},
  title = {M 3 QuestionIng : Multi-modal Multi-span Medical Question Answering},
  journal = {ACM Transactions on Computing for Healthcare},
  year = {2026},
  volume = {7},
  number = {4},
  pages = {1--19},
  doi = {10.1145/3820162}
}
Start for free →