Cross-modal attention model integrating tongue images and descriptions: a novel intelligent TCM approach for pathological organ diagnosis
Frontiers in Physiology · 23.04.2025 · 16
DOI: 10.3389/fphys.2025.1580985 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Physiology, online erschienen am 23.04.2025. Verfasst von Quan Gan und 7 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Engineering, Jiangsu Ocean University, Lianyungang, China. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction
Tongue diagnosis is a fundamental technique in traditional Chinese medicine (TCM), where clinicians evaluate the tongue’s appearance to infer the condition of pathological organs. However, most existing research on intelligent tongue diagnosis primarily focuses on analyzing tongue images, often neglecting the important descriptive text that accompanies these images. This text is an essential component of clinical diagnosis. To overcome this gap, we propose a novel Cross-Modal Pathological Organ Diagnosis Model that integrates tongue images and textual descriptions for more accurate pathological classification Methods Our model extracts features from both the tongue images and the corresponding textual descriptions. These features are then fused using a cross-modal attention mechanism to enhance the classification of pathological organs. The cross-modal attention mechanism enables the model to effectively combine visual and textual information, addressing the limitations of using either modality alone Results We conducted experiments using a self-constructed dataset to evaluate our model’s performance. The results demonstrate that our model outperforms common models regarding overall accuracy. Additionally, ablation studies, where either tongue images or textual descriptions were used alone, confirmed the significant benefit of multimodal fusion in improving diagnostic accuracy.
Discussion
This study introduces a new perspective on intelligent tongue diagnosis in TCM by incorporating visual and textual data. The experimental findings highlight the importance of cross-modal feature fusion for improving the accuracy of pathological diagnosis. Our approach not only contributes to the development of more effective diagnostic systems but also paves the way for future advancements in the automation of TCM diagnosis.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Computer Engineering, Jiangsu Ocean University, Lianyungang, China.
- The Graduate school of East Asian Studies, Yamaguchi University, Yamaguchi-shi, Japan.
- School of Foreign Languages, Jiangsu Ocean University, Lianyungang, China.
Zitieren
APA
Gan, Q., Wang, C., Zhong, Z., Wu, J., Ge, Q., Shi, L., Shang, J., & Liu, C. (2025). Cross-modal attention model integrating tongue images and descriptions: a novel intelligent TCM approach for pathological organ diagnosis. Frontiers in Physiology, 16. https://doi.org/10.3389/fphys.2025.1580985
Vancouver
Gan Q, Wang C, Zhong Z, Wu J, Ge Q, Shi L, et al. Cross-modal attention model integrating tongue images and descriptions: a novel intelligent TCM approach for pathological organ diagnosis. Frontiers in Physiology. 2025;16. doi:10.3389/fphys.2025.1580985
BibTeX
@article{Gan2025,
author = {Gan, Quan and Wang, Chen and Zhong, Zhaoman and Wu, Jiaying and Ge, Qiwei and Shi, Lei and Shang, Jiaqing and Liu, Chuanxia},
title = {Cross-modal attention model integrating tongue images and descriptions: a novel intelligent TCM approach for pathological organ diagnosis},
journal = {Frontiers in Physiology},
year = {2025},
volume = {16},
doi = {10.3389/fphys.2025.1580985}
}
EUVIMED