BI-RADS-compliant structured mammography reporting using locally deployed large language models under privacy constraints
European Radiology · 2025
DOI: 10.1007/s00330-025-12147-2 ↗Kurz gefasst
Multizentrische Studie in der Zeitschrift European Radiology, 2025. Verfasst von Wenjun Sheng und 8 weiteren Autor:innen. Beteiligte Einrichtung: School of Physics and Electronic Engineering, Linyi University, Linyi, China. Hauptthemen laut MEDLINE: Mammography, Breast Neoplasms, Radiology Information Systems, Privacy. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 25 Literaturangaben. Gefördert von: Natural Science Foundation of Shandong Province.
Abstract (Originalsprache)
OBJECTIVE: To develop a privacy-preserving method for structuring free-text mammography reports using a locally fine-tuned, open-source large language model (LLM). MATERIALS AND METHODS: In this multicenter study, 7161 unstructured mammography reports were collected from three institutions. The open-source Llama-3 model was fine-tuned via supervised learning using pseudo-labels from the commercial Qwen-Max model with low-rank adaptation. All labels were pseudo-labels generated by the commercial Qwen-Max model rather than human annotations. Structured outputs followed a BI-RADS-oriented nested JSON schema. Performance was evaluated across 23 features using Precision, Recall, and F1-score. Structural integrity was assessed using the JSON format accuracy (JFA) and field integrity accuracy (FIA) metrics. Statistical comparisons were performed using the paired Wilcoxon signed-rank test and Cohen's d effect size. RESULTS: A total of 7161 reports were retrospectively obtained from three institutions and analyzed. The fine-tuned model achieved strong performance at epoch 10 (Precision 0.942, Recall 0.929, F1-score 0.932), with JFA and FIA reaching 0.964 and 1.000, respectively, showing significant gains over the base model (p 0.8). While slightly below Qwen-Max overall, the model exhibited moderate yet statistically significant differences (p < 0.05; 0.5 < Cohen's d < 0.8), particularly in the "special signs" category (F1 = 0.737 vs 0.947). CONCLUSION: This method effectively converts mammography reports into structured data using a locally fine-tuned, open-source LLM. Although there is a slight performance trade-off, it improves privacy and can be deployed locally. Its accuracy, clinical relevance, and compliance make it a practical solution for medical institutions. KEY POINTS: Question Free-text mammography reports lack standardization, making structured extraction difficult, while existing solutions often compromise privacy, adaptability, or require costly commercial tools. Findings Our fine-tuned LLaMA-3 model achieved high extraction accuracy (F1-score: 0.932) and complete structural integrity (FIA: 1.000) within a fully local deployment pipeline. Clinical relevance This method enables standardized and privacy-preserving mammography reporting without disruption to the clinical workflow, supporting safer AI integration, enhanced data quality, and compliance with regulations such as the General Data Protection Regulation (GDPR).
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Physics and Electronic Engineering, Linyi University, Linyi, China.
- Information Center, Shandong Provincial Maternal and Child Health Care Hospital affiliated to Qingdao University, Jinan, China.
- Department of Radiology, Shandong Provincial Maternal and Child Health Care Hospital affiliated to Qingdao University, Jinan, China.
- Department of Radiology, Jinan Maternal and Child Health Care Hospital affiliated to Shandong First Medical University & Shandong Academy of Medical Sciences, Jinan, China.
- Department of Radiology, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, China.
- School of Physics and Electronic Engineering, Linyi University, Linyi, China. wlyangliu@lyu.edu.cn.
Förderung
- Natural Science Foundation of Shandong Province
Zitieren
APA
Sheng, W., Wang, Y., Xiao, L., Guo, B., Zhang, Y., Qiao, L., Li, Q., Yang, L., & Zhang, Y. (2025). BI-RADS-compliant structured mammography reporting using locally deployed large language models under privacy constraints. European Radiology. https://doi.org/10.1007/s00330-025-12147-2
Vancouver
Sheng W, Wang Y, Xiao L, Guo B, Zhang Y, Qiao L, et al. BI-RADS-compliant structured mammography reporting using locally deployed large language models under privacy constraints. European Radiology. 2025. doi:10.1007/s00330-025-12147-2
BibTeX
@article{Sheng2025,
author = {Sheng, Wenjun and Wang, Yudong and Xiao, Lianxiang and Guo, Bin and Zhang, Yan and Qiao, Liang and Li, Qingqing and Yang, Liu and Zhang, Yi},
title = {BI-RADS-compliant structured mammography reporting using locally deployed large language models under privacy constraints},
journal = {European Radiology},
year = {2025},
doi = {10.1007/s00330-025-12147-2}
}
EUVIMED