Visual-Textual Cross-Modal Interaction Network for Radiology Report Generation
IEEE Signal Processing Letters · 2024 · 31 · 984-988
DOI: 10.1109/lsp.2024.3379005 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Signal Processing Letters, 2024. Verfasst von Wenfeng Zhang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 25 Literaturangaben. Gefördert von: Natural Science Foundation of Chongqing, Science and Technology Research Program of Chongqing Municipal Education Commisson, Chongqing Normal University Foundation.
Abstract (Originalsprache)
The radiology report generation task generates diagnostic descriptions from radiology images, aiming to alleviate the onerous task for radiologists and alerting them to abnormalities. However, the data bias problem poses a persistent challenge, since the abnormal regions usually occupy a small portion of radiology image, while the report generation process should pay greater attention to the abnormal regions. Moreover, the data volume is relatively small compared to large language models, posing challenges during training. To address these issues effectively, we propose a Visual-textual Cross-model Interaction Network (VCIN) to enhance the quality of generated reports. VCIN comprises two key modules: Abundant Clinical Information Embedding (ACIE), which gathers rich cross-modal interaction information to promote the report generation of abnormal regions; and a Bert-based Decoder-only Generator (BDG), built on Bert architecture to mitigate training difficulties. The superior performance of our proposed model is demonstrated through experimental results obtained from two public benchmark datasets.
Förderung
- Natural Science Foundation of Chongqing
- Science and Technology Research Program of Chongqing Municipal Education Commisson
- Chongqing Normal University Foundation
- National Natural Science Foundation of China
- Natural Science Foundation of Shandong Province
- Natural Science Foundation Innovation and Development Joint Fund Project of Chongqing
Zitieren
APA
Zhang, W., Cai, B., Hu, J., Qin, Q., & Xie, K. (2024). Visual-Textual Cross-Modal Interaction Network for Radiology Report Generation. IEEE Signal Processing Letters, 31, 984-988. https://doi.org/10.1109/lsp.2024.3379005
Vancouver
Zhang W, Cai B, Hu J, Qin Q, Xie K. Visual-Textual Cross-Modal Interaction Network for Radiology Report Generation. IEEE Signal Processing Letters. 2024;31:984-988. doi:10.1109/lsp.2024.3379005
BibTeX
@article{Zhang2024,
author = {Zhang, Wenfeng and Cai, Baoning and Hu, Jianming and Qin, Qibing and Xie, Kezhen},
title = {Visual-Textual Cross-Modal Interaction Network for Radiology Report Generation},
journal = {IEEE Signal Processing Letters},
year = {2024},
volume = {31},
pages = {984--988},
doi = {10.1109/lsp.2024.3379005}
}
EUVIMED