EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

Long Document Classification in the Transformer Era: A Survey on Challenges, Advances, and Open Issues

Renzo Alva Principe, Nicola Chiarini, Marco Viviani

WIREs Data Mining and Knowledge Discovery · 2025-06 · 15 (2)

DOI: 10.1002/widm.70019 ↗

At a glance

Journal article in WIREs Data Mining and Knowledge Discovery, published online on 2025-05-08. Written by Renzo Alva Principe and 2 co-authors. Cited 14 times so far (Crossref); 119 references. The full text is freely available (open access, CC BY 4.0).

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Automatic Document Classification (ADC) refers to the process of automatically categorizing or labeling documents into predefined classes or categories. Its effectiveness may depend on various factors, including the models used for the formal representation of documents, the classification techniques applied, or a combination of both. Recently, Transformer models have gained popularity due to their pre‐training on large corpora, allowing for flexible knowledge transfer to downstream tasks, such as ADC. However, such models can face challenges when handling “long” documents, particularly due to input sequence length constraints, which can have knock‐on effects on the task we refer to as Automatic Long Document Classification (ALDC). Distinct models for tackling this limitation of Transformers have been proposed over the past few years, and employed to perform ALDC; however, their application to this task has resulted in some inconsistent outcomes, struggles to surpass simple baselines, and difficulties in generalizing across diverse datasets and scenarios. That is why this survey aims to illustrate these limitations, by: (i) presenting current long document representation issues and solutions proposed in the literature; (ii) based on such solutions, illustrating a comprehensive analysis of their application in ALDC and their effectiveness; and (iii) discussing current evaluation strategies in ALDC with particular reference to suitable baselines and actual long‐document benchmark datasets.

Cite

APA

Principe, R. A., Chiarini, N., & Viviani, M. (2025). Long Document Classification in the Transformer Era: A Survey on Challenges, Advances, and Open Issues. WIREs Data Mining and Knowledge Discovery, 15(2). https://doi.org/10.1002/widm.70019

Vancouver

Principe RA, Chiarini N, Viviani M. Long Document Classification in the Transformer Era: A Survey on Challenges, Advances, and Open Issues. WIREs Data Mining and Knowledge Discovery. 2025;15(2). doi:10.1002/widm.70019

BibTeX
@article{Principe2025,
  author = {Principe, Renzo Alva and Chiarini, Nicola and Viviani, Marco},
  title = {Long Document Classification in the Transformer Era: A Survey on Challenges, Advances, and Open Issues},
  journal = {WIREs Data Mining and Knowledge Discovery},
  year = {2025},
  volume = {15},
  number = {2},
  doi = {10.1002/widm.70019}
}
Start for free →