EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Unified Multi-Reference Framework for Training and Evaluation in Abstractive Summarization

Abishek B. Rao, Shivani G. Aithal, Sanjay Singh

IEEE Access · 2026 · 14 · 98667-98690

DOI: 10.1109/access.2026.3686976 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Abishek B. Rao und 2 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Abstractive summarization models rely on a single gold reference for training and evaluation, assuming it is optimal. We show that machine-generated summaries frequently exhibit stronger semantic alignment with source articles than gold references on CNN/DM and XSum. To address this, we propose a unified framework comprising the Multi-Reference Training Framework (MRTF) and Multi-Reference Evaluation Framework (MREF), both driven by an LSI-based Semantic Selection Mechanism (LSI-SSM). Grounded in the Eckart–Young–Mirsky theorem, LSI-SSM uses only $k{=}2$ dimensions to achieve 98.9% of neural embedding quality at zero GPU cost. Across BRIO, BART, PEGASUS, and DistilBART, MRTF yields statistically significant ( $p \lt 0.001$ ) ROUGE-1 gains of +2.13 to +2.61 on CNN/DM. MREF reveals that traditional evaluation undervalues summaries by 3–5 ROUGE-1 points, with distributional bias analysis confirming 83–87% of corrections reflect genuine quality. Cross-domain validation on Multi-News, SAMSum, and PubMed confirms generalizability.

Zitieren

APA

Rao, A. B., Aithal, S. G., & Singh, S. (2026). A Unified Multi-Reference Framework for Training and Evaluation in Abstractive Summarization. IEEE Access, 14, 98667-98690. https://doi.org/10.1109/access.2026.3686976

Vancouver

Rao AB, Aithal SG, Singh S. A Unified Multi-Reference Framework for Training and Evaluation in Abstractive Summarization. IEEE Access. 2026;14:98667-98690. doi:10.1109/access.2026.3686976

BibTeX
@article{Rao2026,
  author = {Rao, Abishek B. and Aithal, Shivani G. and Singh, Sanjay},
  title = {A Unified Multi-Reference Framework for Training and Evaluation in Abstractive Summarization},
  journal = {IEEE Access},
  year = {2026},
  volume = {14},
  pages = {98667--98690},
  doi = {10.1109/access.2026.3686976}
}
Kostenlos starten →