EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

TongueNet: a multi-modal fusion and multi-label classification model for traditional Chinese Medicine tongue diagnosis

Lijuan Yang, Qiumei Dong, Da Lin, Xinliang Lü

Frontiers in Physiology · 25.04.2025 · 16

DOI: 10.3389/fphys.2025.1527751 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Physiology, online erschienen am 25.04.2025. Verfasst von Lijuan Yang und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Rheumatology, Inner Mongolia Autonomous Region Hospital of Traditional Chinese Medicine, Hohhot, China. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Tongue diagnosis in Traditional Chinese Medicine (TCM) plays a crucial role in clinical practice. By observing the shape, color, and coating of the tongue, practitioners can assist in determining the nature and location of a disease. However, the field of tongue diagnosis currently faces challenges such as data scarcity and a lack of efficient multimodal diagnostic models, making it difficult to fully align with TCM theories and clinical needs. Additionally, existing methods generally lack multi-label classification capabilities, making it challenging to simultaneously meet the multidimensional requirements of TCM diagnosis for disease nature and location. To address these issues, this paper proposes TongueNet, a multimodal deep learning model that integrates tongue image data with text-based features. The model utilizes a Hierarchical Aggregation Network (HAN) and a Feature Space Projection Module to efficiently extract and fuse features while introducing consistency and complementarity constraints to optimize multimodal information fusion. Furthermore, the model incorporates a multi-scale attention mechanism (EMA) to enhance the diversity and accuracy of feature weighting and employs a Kolmogorov-Arnold Network (KAN) instead of traditional MLPs for output optimization, thereby improving the representation of complex features. For model training, this study integrates three publicly available tongue image datasets from the Roboflow platform and enlists multiple experts for multimodal annotation, incorporating multi-label information on disease nature and location to align with TCM clinical needs. Experimental results demonstrate that TongueNet outperforms existing models in both disease nature and disease location classification tasks. Specifically, in the disease nature classification task, it achieves 89.12% accuracy and an AUC of 83%; in the disease location classification task, it achieves 86.47% accuracy and an AUC of 81%. Moreover, TongueNet contains only 32.1 M parameters, significantly reducing computational resource requirements while maintaining high diagnostic performance. TongueNet provides a new approach for the intelligent development of TCM tongue diagnosis.

Schlagwörter

Stichwörter der Autor:innen

deep learningmulti-label classificationmultimodal fusiontongue diagnosistraditional Chinese Medicine (TCM)

Autor:innen und Institutionen

  • Department of Rheumatology, Inner Mongolia Autonomous Region Hospital of Traditional Chinese Medicine, Hohhot, China.
  • College of Traditional Chinese Medicine, Inner Mongolia Medical University, Hohhot, China.
  • School of Mathematical Sciences, Inner Mongolia University, Hohhot, China.

Zitieren

APA

Yang, L., Dong, Q., Lin, D., & Lü, X. (2025). TongueNet: a multi-modal fusion and multi-label classification model for traditional Chinese Medicine tongue diagnosis. Frontiers in Physiology, 16. https://doi.org/10.3389/fphys.2025.1527751

Vancouver

Yang L, Dong Q, Lin D, Lü X. TongueNet: a multi-modal fusion and multi-label classification model for traditional Chinese Medicine tongue diagnosis. Frontiers in Physiology. 2025;16. doi:10.3389/fphys.2025.1527751

BibTeX
@article{Yang2025,
  author = {Yang, Lijuan and Dong, Qiumei and Lin, Da and Lü, Xinliang},
  title = {TongueNet: a multi-modal fusion and multi-label classification model for traditional Chinese Medicine tongue diagnosis},
  journal = {Frontiers in Physiology},
  year = {2025},
  volume = {16},
  doi = {10.3389/fphys.2025.1527751}
}
Kostenlos starten →