EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation

He Zhu iD, Ren Togo iD, Takahiro Ogawa iD, Miki Haseyama iD

Sensors · 2023

DOI: 10.3390/s23031057 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2023. Verfasst von He Zhu und 3 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan. Hauptthemen laut MEDLINE: Natural Language Processing, Semantics. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 58 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: JSPS KAKENHI.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Auxiliary clinical diagnosis has been researched to solve unevenly and insufficiently distributed clinical resources. However, auxiliary diagnosis is still dominated by human physicians, and how to make intelligent systems more involved in the diagnosis process is gradually becoming a concern. An interactive automated clinical diagnosis with a question-answering system and a question generation system can capture a patient’s conditions from multiple perspectives with less physician involvement by asking different questions to drive and guide the diagnosis. This clinical diagnosis process requires diverse information to evaluate a patient from different perspectives to obtain an accurate diagnosis. Recently proposed medical question generation systems have not considered diversity. Thus, we propose a diversity learning-based visual question generation model using a multi-latent space to generate informative question sets from medical images. The proposed method generates various questions by embedding visual and language information in different latent spaces, whose diversity is trained by our newly proposed loss. We have also added control over the categories of generated questions, making the generated questions directional. Furthermore, we use a new metric named similarity to accurately evaluate the proposed model’s performance. The experimental results on the Slake and VQA-RAD datasets demonstrate that the proposed method can generate questions with diverse information. Our model works with an answering model for interactive automated clinical diagnosis and generates datasets to replace the process of annotation that incurs huge labor costs.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingSemanticsHumansLanguage

Stichwörter der Autor:innen

computer visionmedical image analysismedical informaticsnatural language processingvisual question generation

Autor:innen und Institutionen

  • Graduate School of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan.
  • Faculty of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan.

ORCID-Profile: He Zhu, Ren Togo, Takahiro Ogawa, Miki Haseyama

Förderung

  • JSPS KAKENHI

Zitieren

APA

Zhu, H., Togo, R., Ogawa, T., & Haseyama, M. (2023). Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation. Sensors. https://doi.org/10.3390/s23031057

Vancouver

Zhu H, Togo R, Ogawa T, Haseyama M. Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation. Sensors. 2023. doi:10.3390/s23031057

BibTeX
@article{Zhu2023,
  author = {Zhu, He and Togo, Ren and Ogawa, Takahiro and Haseyama, Miki},
  title = {Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation},
  journal = {Sensors},
  year = {2023},
  doi = {10.3390/s23031057}
}
Kostenlos starten →