EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Cross-modal attention model integrating tongue images and descriptions: a novel intelligent TCM approach for pathological organ diagnosis

Quan Gan, Chen Wang, Zhaoman Zhong, Jiaying Wu, Qiwei Ge, Lei Shi, Jiaqing Shang, Chuanxia Liu

Frontiers in Physiology · 23.04.2025 · 16

DOI: 10.3389/fphys.2025.1580985 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Physiology, online erschienen am 23.04.2025. Verfasst von Quan Gan und 7 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Engineering, Jiangsu Ocean University, Lianyungang, China. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Tongue diagnosis is a fundamental technique in traditional Chinese medicine (TCM), where clinicians evaluate the tongue’s appearance to infer the condition of pathological organs. However, most existing research on intelligent tongue diagnosis primarily focuses on analyzing tongue images, often neglecting the important descriptive text that accompanies these images. This text is an essential component of clinical diagnosis. To overcome this gap, we propose a novel Cross-Modal Pathological Organ Diagnosis Model that integrates tongue images and textual descriptions for more accurate pathological classification Methods Our model extracts features from both the tongue images and the corresponding textual descriptions. These features are then fused using a cross-modal attention mechanism to enhance the classification of pathological organs. The cross-modal attention mechanism enables the model to effectively combine visual and textual information, addressing the limitations of using either modality alone Results We conducted experiments using a self-constructed dataset to evaluate our model’s performance. The results demonstrate that our model outperforms common models regarding overall accuracy. Additionally, ablation studies, where either tongue images or textual descriptions were used alone, confirmed the significant benefit of multimodal fusion in improving diagnostic accuracy.

Discussion

This study introduces a new perspective on intelligent tongue diagnosis in TCM by incorporating visual and textual data. The experimental findings highlight the importance of cross-modal feature fusion for improving the accuracy of pathological diagnosis. Our approach not only contributes to the development of more effective diagnostic systems but also paves the way for future advancements in the automation of TCM diagnosis.

Schlagwörter

Stichwörter der Autor:innen

cross-modal attentionpathological organtextual descriptionstongue diagnosistongue images analysis

Autor:innen und Institutionen

  • School of Computer Engineering, Jiangsu Ocean University, Lianyungang, China.
  • The Graduate school of East Asian Studies, Yamaguchi University, Yamaguchi-shi, Japan.
  • School of Foreign Languages, Jiangsu Ocean University, Lianyungang, China.

Zitieren

APA

Gan, Q., Wang, C., Zhong, Z., Wu, J., Ge, Q., Shi, L., Shang, J., & Liu, C. (2025). Cross-modal attention model integrating tongue images and descriptions: a novel intelligent TCM approach for pathological organ diagnosis. Frontiers in Physiology, 16. https://doi.org/10.3389/fphys.2025.1580985

Vancouver

Gan Q, Wang C, Zhong Z, Wu J, Ge Q, Shi L, et al. Cross-modal attention model integrating tongue images and descriptions: a novel intelligent TCM approach for pathological organ diagnosis. Frontiers in Physiology. 2025;16. doi:10.3389/fphys.2025.1580985

BibTeX
@article{Gan2025,
  author = {Gan, Quan and Wang, Chen and Zhong, Zhaoman and Wu, Jiaying and Ge, Qiwei and Shi, Lei and Shang, Jiaqing and Liu, Chuanxia},
  title = {Cross-modal attention model integrating tongue images and descriptions: a novel intelligent TCM approach for pathological organ diagnosis},
  journal = {Frontiers in Physiology},
  year = {2025},
  volume = {16},
  doi = {10.3389/fphys.2025.1580985}
}
Kostenlos starten →