EUVIMEDEUVIMED entdecken
Crossref

Diagnosing Rare Genetic Diseases From Facial Images Using Visual Prompting With Multimodal Large Language Models: Diagnostic Accuracy Study

Jie Song iD, Zhichuan Xu iD, Meng Xiao iD, Cheng Bi iD, Yuxin Zhang iD, Xin Zheng iD, Xiaoran Li iD, Ziyu Lu iD, Qiongfang Cao iD, Bairong Shen iD

JMIR Medical Informatics · 02.11.2026 · 14 · e105860

DOI: 10.2196/105860 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JMIR Medical Informatics, online erschienen am 02.11.2026. Verfasst von Jie Song und 9 weiteren Autor:innen. Beteiligte Einrichtung: Department of Dermatovenereology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular, West China Hospital of …. Hauptthemen laut MEDLINE: Rare Diseases, Face. Die Arbeit nennt 42 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Rare genetic diseases often present with recognizable facial phenotypes. Whether multimodal large language models (MLLMs) can prioritize the diagnosis of rare genetic diseases from facial images and whether image enhancement or visual prompting can improve this process remain unclear.

Objective

This study aimed to evaluate the diagnostic accuracy of MLLMs for rare genetic diseases using facial images and to assess whether super-resolution enhancement or red-circle visual prompting improved performance.

Methods

This diagnostic accuracy study used 4746 facial images from the GestaltMatcher Database (GMDB), including 882 images in GMDB-Frequent from diseases with at least 6 images and 3864 images in GMDB-Rare from diseases with fewer than 6 images. Three locally deployed open-weight MLLMs—Gemma-4-31B, Qwen3.6-35B-A3B, and Qwen3.8-27B—generated 50 ranked candidate diagnoses under raw image, CodeFormer-enhanced, Qwen-Image-Edit-2511–generated red-circle–prompted, mismatched red-circle, and 2-stage image-to-phenotype conditions. Candidate diagnoses were mapped to Mondo Disease Ontology (MONDO) identifiers. In addition, we evaluated a random baseline, visual-prompt quality, expert-annotated regions, repeated-inference stability, model-reported confidence, diagnostic-list perplexity, prediction overlap, and disease composition.

Results

Top-50 accuracy ranged from 13.95% to 18.25% in GMDB-Frequent and from 0.57% to 1.35% in GMDB-Rare. Red-circle prompting produced numerical accuracy increases in 5 of 6 model-subset comparisons. Qwen3.8-27B showed a significant improvement in GMDB-Rare, with accuracy increasing from 0.57% to 1.27% (adjusted P=.009). In the good visual-prompt subgroup of GMDB-Frequent, Gemma-4-31B showed significant improvements over the super-resolution condition in both physician-rated analyses, with accuracy increases of 5.76 percentage points (Holm-adjusted P=.005) and 4.46 percentage points (Holm-adjusted P=.048), respectively. Matched red-circle inputs yielded higher accuracy than mismatched inputs across all model-subset comparisons, with adjusted P>.05. The image-to-phenotype workflow improved accuracy for Gemma-4-31B in GMDB-Frequent and Qwen3.8-27B in GMDB-Rare. In GMDB-Rare, the random baseline achieved 8.15% accuracy and exceeded all 3 models under the raw image and red-circle conditions. Prediction sets remained substantially overlapping after patient-image replacement, and top-1 predictions concentrated on a small number of diseases.

Conclusions

Under the evaluated experimental settings, the 3 MLLMs showed limited ability to diagnose rare genetic diseases from facial images, particularly diseases in GMDB-Rare. Visual prompting and image-to-phenotype diagnosis produced benefits in specific models and subgroups. The prediction-overlap, mismatched-image, and disease-composition analyses indicate that model outputs were only weakly constrained by patient-specific facial information. Further studies should evaluate physician comparisons, prospective multicenter data, and phenotype-grounded multimodal diagnostic workflows.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Rare DiseasesFaceLarge Language ModelsHumansPhenotype

Stichwörter der Autor:innen

AIartificial intelligencefacial phenotypesmultimodal large language modelsrare genetic diseasesvisual prompting

Autor:innen und Institutionen

  • Department of Dermatovenereology and Institutes for Systems Genetics, Frontiers Science Center for Disease-related Molecular, West China Hospital of Sichuan University, Chengdu, Sichuan, China.

ORCID-Profile: Jie Song, Zhichuan Xu, Meng Xiao, Cheng Bi, Yuxin Zhang, Xin Zheng, Xiaoran Li, Ziyu Lu, Qiongfang Cao, Bairong Shen

Zitieren

APA

Song, J., Xu, Z., Xiao, M., Bi, C., Zhang, Y., Zheng, X., Li, X., Lu, Z., Cao, Q., & Shen, B. (2026). Diagnosing Rare Genetic Diseases From Facial Images Using Visual Prompting With Multimodal Large Language Models: Diagnostic Accuracy Study. JMIR Medical Informatics, 14, e105860. https://doi.org/10.2196/105860

Vancouver

Song J, Xu Z, Xiao M, Bi C, Zhang Y, Zheng X, et al. Diagnosing Rare Genetic Diseases From Facial Images Using Visual Prompting With Multimodal Large Language Models: Diagnostic Accuracy Study. JMIR Medical Informatics. 2026;14:e105860. doi:10.2196/105860

BibTeX
@article{Song2026,
  author = {Song, Jie and Xu, Zhichuan and Xiao, Meng and Bi, Cheng and Zhang, Yuxin and Zheng, Xin and Li, Xiaoran and Lu, Ziyu and Cao, Qiongfang and Shen, Bairong},
  title = {Diagnosing Rare Genetic Diseases From Facial Images Using Visual Prompting With Multimodal Large Language Models: Diagnostic Accuracy Study},
  journal = {JMIR Medical Informatics},
  year = {2026},
  volume = {14},
  pages = {e105860},
  doi = {10.2196/105860}
}
Kostenlos starten →