Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation
Sensors · 2023
DOI: 10.3390/s23031057 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2023. Verfasst von He Zhu und 3 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan. Hauptthemen laut MEDLINE: Natural Language Processing, Semantics. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 58 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: JSPS KAKENHI.
Abstract (Originalsprache)
Auxiliary clinical diagnosis has been researched to solve unevenly and insufficiently distributed clinical resources. However, auxiliary diagnosis is still dominated by human physicians, and how to make intelligent systems more involved in the diagnosis process is gradually becoming a concern. An interactive automated clinical diagnosis with a question-answering system and a question generation system can capture a patient’s conditions from multiple perspectives with less physician involvement by asking different questions to drive and guide the diagnosis. This clinical diagnosis process requires diverse information to evaluate a patient from different perspectives to obtain an accurate diagnosis. Recently proposed medical question generation systems have not considered diversity. Thus, we propose a diversity learning-based visual question generation model using a multi-latent space to generate informative question sets from medical images. The proposed method generates various questions by embedding visual and language information in different latent spaces, whose diversity is trained by our newly proposed loss. We have also added control over the categories of generated questions, making the generated questions directional. Furthermore, we use a new metric named similarity to accurately evaluate the proposed model’s performance. The experimental results on the Slake and VQA-RAD datasets demonstrate that the proposed method can generate questions with diverse information. Our model works with an answering model for interactive automated clinical diagnosis and generates datasets to replace the process of annotation that incurs huge labor costs.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Graduate School of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan.
- Faculty of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan.
ORCID-Profile: He Zhu, Ren Togo, Takahiro Ogawa, Miki Haseyama
Förderung
- JSPS KAKENHI
Zitieren
APA
Zhu, H., Togo, R., Ogawa, T., & Haseyama, M. (2023). Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation. Sensors. https://doi.org/10.3390/s23031057
Vancouver
Zhu H, Togo R, Ogawa T, Haseyama M. Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation. Sensors. 2023. doi:10.3390/s23031057
BibTeX
@article{Zhu2023,
author = {Zhu, He and Togo, Ren and Ogawa, Takahiro and Haseyama, Miki},
title = {Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation},
journal = {Sensors},
year = {2023},
doi = {10.3390/s23031057}
}
EUVIMED