EUVIMEDEUVIMED entdecken
Crossref

Spatio-Temporal and Retrieval-Augmented Modeling for Chest X-Ray Report Generation

Yan Yang, Xiaoxing You, Ke Zhang, Zhenqi Fu, Xianyun Wang, Jiajun Ding, Jiamei Sun, Zhou Yu, Qingming Huang, Weidong Han, Jun Yu

IEEE Transactions on Medical Imaging · 2025-07 · 44 (7) · 2892-2905

DOI: 10.1109/tmi.2025.3554498 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Medical Imaging, 2025-07. Verfasst von Yan Yang und 10 weiteren Autor:innen. Hauptthemen laut MEDLINE: Radiography, Thoracic, Radiographic Image Interpretation, Computer-Assisted, Image Processing, Computer-Assisted. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 62 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Zhejiang Provincial Natural Science Foundation of China, Central Government Guiding Fund for Local Science and Technology Development.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Chest X-ray report generation has attracted increasing research attention. However, most existing methods neglect the temporal information and typically generate reports conditioned on a fixed number of images. In this paper, we propose STREAM: Spatio-Temporal and REtrieval-Augmented Modelling for automatic chest X-ray report generation. It mimics clinical diagnosis by integrating current and historical studies to interpret the present condition (temporal), with each study containing images from multi-views (spatial). Concretely, our STREAM is built upon an encoder-decoder architecture, utilizing a large language model (LLM) as the decoder. Overall, spatio-temporal visual dynamics are packed as visual prompts and regional semantic entities are retrieved as textual prompts. First, a token packer is proposed to capture condensed spatio-temporal visual dynamics, enabling the flexible fusion of images from current and historical studies. Second, to augment the generation with existing knowledge and regional details, a progressive semantic retriever is proposed to retrieve semantic entities from a preconstructed knowledge bank as heuristic text prompts. The knowledge bank is constructed to encapsulate anatomical chest X-ray knowledge into structured entities, each linked to a specific chest region. Extensive experiments on public datasets have shown the state-of-the-art performance of our method. Related codes and the knowledge bank are available at https://github.com/yangyan22/STREAM.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Radiography, ThoracicRadiographic Image Interpretation, Computer-AssistedImage Processing, Computer-AssistedHumansAlgorithmsDatabases, Factual

Förderung

  • National Natural Science Foundation of China
  • Zhejiang Provincial Natural Science Foundation of China
  • Central Government Guiding Fund for Local Science and Technology Development

Zitieren

APA

Yang, Y., You, X., Zhang, K., Fu, Z., Wang, X., Ding, J., Sun, J., Yu, Z., Huang, Q., Han, W., & Yu, J. (2025). Spatio-Temporal and Retrieval-Augmented Modeling for Chest X-Ray Report Generation. IEEE Transactions on Medical Imaging, 44(7), 2892-2905. https://doi.org/10.1109/tmi.2025.3554498

Vancouver

Yang Y, You X, Zhang K, Fu Z, Wang X, Ding J, et al. Spatio-Temporal and Retrieval-Augmented Modeling for Chest X-Ray Report Generation. IEEE Transactions on Medical Imaging. 2025;44(7):2892-2905. doi:10.1109/tmi.2025.3554498

BibTeX
@article{Yang2025,
  author = {Yang, Yan and You, Xiaoxing and Zhang, Ke and Fu, Zhenqi and Wang, Xianyun and Ding, Jiajun and Sun, Jiamei and Yu, Zhou and Huang, Qingming and Han, Weidong and Yu, Jun},
  title = {Spatio-Temporal and Retrieval-Augmented Modeling for Chest X-Ray Report Generation},
  journal = {IEEE Transactions on Medical Imaging},
  year = {2025},
  volume = {44},
  number = {7},
  pages = {2892--2905},
  doi = {10.1109/tmi.2025.3554498}
}
Kostenlos starten →