MMFVS-Net: A triple-symmetric cross-attention network for multimodal optical image fusion and high-accuracy virtual staining of breast cancer tissues
Computer Methods and Programs in Biomedicine · 2026-10 · 285 · 109536
DOI: 10.1016/j.cmpb.2026.109536 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computer Methods and Programs in Biomedicine, 2026-10. Verfasst von Lin Wei Shang und 9 weiteren Autor:innen. Beteiligte Einrichtung: College of automation engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China. Electronic address: Shang@nuaa.edu.cn. Hauptthemen laut MEDLINE: Breast Neoplasms, Multimodal Imaging, Staining and Labeling, Optical Imaging. Die Arbeit nennt 35 Literaturangaben. Gefördert von: Jiangsu Provincial Key Research and Development Program, National Natural Science Foundation of China.
Abstract (Originalsprache)
BACKGROUND AND OBJECTIVE Virtual staining is based on label-free imaging technology and deep learning, which enables the efficient and non-destructive generation of virtual stained images with effects similar to those of traditional pathological staining, thus holding great application potential in future clinical medical practice. However, the information provided by a single imaging technology is limited, which is insufficient to support high-accuracy virtual staining. METHODS This study took breast cancer tissue as the research object. Bright-field, auto-fluorescence and phase contrast imaging technology were introduced to simultaneously extract the multimodal complementary information. A multi-branch input network was established to fuse the three types of optical images and generate high-accuracy virtual stained images. To enhance the model's performance, three strategies were employed, including designing a Triple-symmetric cross-attention module to handle the interactive information of multimodal image features, introducing multi-scale discriminator to ensure the model's ability to capture both global features and local details, and applying transfer-learning to address the gradient dilution issue of multi-branches. RESULTS Through the evaluation of various image similarity metrics and nuclear statistical metrics, the virtual stained images based on multimodal fusion achieved high accuracy and were significantly superior to any single-modal virtual staining approach. Further ablation experiments explained the reason of the aforementioned results and quantitatively demonstrated the improvements brought by the three optimization strategies. CONCLUSIONS This work advances the application of multimodal fusion in virtual staining, address the key challenges of cross-modal interactive information mining and multi-branch model training, and further enhance the accuracy and reliability of virtual staining.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- College of automation engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China. Electronic address: Shang@nuaa.edu.cn.
- College of automation engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China.
- Department of Pathology, Jiangsu Cancer Hospital, Jiangsu Institute of Cancer Research & the Affiliated Cancer Hospital of Nanjing Medical University, Nanjing, 210016, China.
- College of automation engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China. Electronic address: wanghuijie@nuaa.edu.cn.
- College of automation engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China. Electronic address: yin@nuaa.edu.cn.
Förderung
- Jiangsu Provincial Key Research and Development Program
- National Natural Science Foundation of China
Zitieren
APA
Shang, L. W., Guo, Y. X., Li, Y. H., Zhu, J. B., Ji, X. Y., Xiong, S., Wu, J. J., Huang, X., Wang, H. J., & Yin, J. H. (2026). MMFVS-Net: A triple-symmetric cross-attention network for multimodal optical image fusion and high-accuracy virtual staining of breast cancer tissues. Computer Methods and Programs in Biomedicine, 285, 109536. https://doi.org/10.1016/j.cmpb.2026.109536
Vancouver
Shang LW, Guo YX, Li YH, Zhu JB, Ji XY, Xiong S, et al. MMFVS-Net: A triple-symmetric cross-attention network for multimodal optical image fusion and high-accuracy virtual staining of breast cancer tissues. Computer Methods and Programs in Biomedicine. 2026;285:109536. doi:10.1016/j.cmpb.2026.109536
BibTeX
@article{Shang2026,
author = {Shang, Lin Wei and Guo, Ying Xi and Li, Yun Hong and Zhu, Jian Bo and Ji, Xin Yi and Xiong, Shuang and Wu, Jin Jin and Huang, Xing and Wang, Hui Jie and Yin, Jian Hua},
title = {MMFVS-Net: A triple-symmetric cross-attention network for multimodal optical image fusion and high-accuracy virtual staining of breast cancer tissues},
journal = {Computer Methods and Programs in Biomedicine},
year = {2026},
volume = {285},
pages = {109536},
doi = {10.1016/j.cmpb.2026.109536}
}
EUVIMED