EUVIMEDEUVIMED entdecken
Crossref

MMFVS-Net: A triple-symmetric cross-attention network for multimodal optical image fusion and high-accuracy virtual staining of breast cancer tissues

Lin Wei Shang, Ying Xi Guo, Yun Hong Li, Jian Bo Zhu, Xin Yi Ji, Shuang Xiong, Jin Jin Wu, Xing Huang, Hui Jie Wang, Jian Hua Yin

Computer Methods and Programs in Biomedicine · 2026-10 · 285 · 109536

DOI: 10.1016/j.cmpb.2026.109536 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computer Methods and Programs in Biomedicine, 2026-10. Verfasst von Lin Wei Shang und 9 weiteren Autor:innen. Beteiligte Einrichtung: College of automation engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China. Electronic address: Shang@nuaa.edu.cn. Hauptthemen laut MEDLINE: Breast Neoplasms, Multimodal Imaging, Staining and Labeling, Optical Imaging. Die Arbeit nennt 35 Literaturangaben. Gefördert von: Jiangsu Provincial Key Research and Development Program, National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

BACKGROUND AND OBJECTIVE Virtual staining is based on label-free imaging technology and deep learning, which enables the efficient and non-destructive generation of virtual stained images with effects similar to those of traditional pathological staining, thus holding great application potential in future clinical medical practice. However, the information provided by a single imaging technology is limited, which is insufficient to support high-accuracy virtual staining. METHODS This study took breast cancer tissue as the research object. Bright-field, auto-fluorescence and phase contrast imaging technology were introduced to simultaneously extract the multimodal complementary information. A multi-branch input network was established to fuse the three types of optical images and generate high-accuracy virtual stained images. To enhance the model's performance, three strategies were employed, including designing a Triple-symmetric cross-attention module to handle the interactive information of multimodal image features, introducing multi-scale discriminator to ensure the model's ability to capture both global features and local details, and applying transfer-learning to address the gradient dilution issue of multi-branches. RESULTS Through the evaluation of various image similarity metrics and nuclear statistical metrics, the virtual stained images based on multimodal fusion achieved high accuracy and were significantly superior to any single-modal virtual staining approach. Further ablation experiments explained the reason of the aforementioned results and quantitatively demonstrated the improvements brought by the three optimization strategies. CONCLUSIONS This work advances the application of multimodal fusion in virtual staining, address the key challenges of cross-modal interactive information mining and multi-branch model training, and further enhance the accuracy and reliability of virtual staining.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Breast NeoplasmsMultimodal ImagingStaining and LabelingOptical ImagingImage Processing, Computer-AssistedHumansFemaleAlgorithmsDeep LearningNeural Networks, Computer

Stichwörter der Autor:innen

Breast cancerDeep learningMultimodal fusionVirtual staining

Autor:innen und Institutionen

  • College of automation engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China. Electronic address: Shang@nuaa.edu.cn.
  • College of automation engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China.
  • Department of Pathology, Jiangsu Cancer Hospital, Jiangsu Institute of Cancer Research & the Affiliated Cancer Hospital of Nanjing Medical University, Nanjing, 210016, China.
  • College of automation engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China. Electronic address: wanghuijie@nuaa.edu.cn.
  • College of automation engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China. Electronic address: yin@nuaa.edu.cn.

Förderung

  • Jiangsu Provincial Key Research and Development Program
  • National Natural Science Foundation of China

Zitieren

APA

Shang, L. W., Guo, Y. X., Li, Y. H., Zhu, J. B., Ji, X. Y., Xiong, S., Wu, J. J., Huang, X., Wang, H. J., & Yin, J. H. (2026). MMFVS-Net: A triple-symmetric cross-attention network for multimodal optical image fusion and high-accuracy virtual staining of breast cancer tissues. Computer Methods and Programs in Biomedicine, 285, 109536. https://doi.org/10.1016/j.cmpb.2026.109536

Vancouver

Shang LW, Guo YX, Li YH, Zhu JB, Ji XY, Xiong S, et al. MMFVS-Net: A triple-symmetric cross-attention network for multimodal optical image fusion and high-accuracy virtual staining of breast cancer tissues. Computer Methods and Programs in Biomedicine. 2026;285:109536. doi:10.1016/j.cmpb.2026.109536

BibTeX
@article{Shang2026,
  author = {Shang, Lin Wei and Guo, Ying Xi and Li, Yun Hong and Zhu, Jian Bo and Ji, Xin Yi and Xiong, Shuang and Wu, Jin Jin and Huang, Xing and Wang, Hui Jie and Yin, Jian Hua},
  title = {MMFVS-Net: A triple-symmetric cross-attention network for multimodal optical image fusion and high-accuracy virtual staining of breast cancer tissues},
  journal = {Computer Methods and Programs in Biomedicine},
  year = {2026},
  volume = {285},
  pages = {109536},
  doi = {10.1016/j.cmpb.2026.109536}
}
Kostenlos starten →