Digital forgetting in large language models: a survey of unlearning methods
Artificial Intelligence Review · 13.01.2025 · 58 (3)
DOI: 10.1007/s10462-024-11078-6 ↗Kurz gefasst
Fachartikel in der Zeitschrift Artificial Intelligence Review, online erschienen am 13.01.2025. Verfasst von Alberto Blanco-Justicia und 6 weiteren Autor:innen. Die Arbeit wurde bisher 35-mal zitiert (Crossref) und nennt 121 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Universitat Rovira i Virgili.
Abstract (Originalsprache)
Large language models (LLMs) have become the state of the art in natural language processing. The massive adoption of generative LLMs and the capabilities they have shown have prompted public concerns regarding their impact on the labor market, privacy, the use of copyrighted work, and how these models align with human ethics and the rule of law. As a response, new regulations are being pushed, which require developers and service providers to evaluate, monitor, and forestall or at least mitigate the risks posed by their models. One mitigation strategy is digital forgetting: given a model with undesirable knowledge or behavior, the goal is to obtain a new model where the detected issues are no longer present. Digital forgetting is usually enforced via machine unlearning techniques, which modify trained machine learning models for them to behave as models trained on a subset of the original training data. In this work, we describe the motivations and desirable properties of digital forgetting when applied to LLMs, and we survey recent works on machine unlearning. Specifically, we propose a taxonomy of unlearning methods based on the reach and depth of the modifications done on the models, we discuss and compare the effectiveness of machine unlearning methods for LLMs proposed so far, and we survey their evaluation. Finally, we describe open problems of machine unlearning applied to LLMs and we put forward recommendations for developers and practitioners.
Förderung
- Universitat Rovira i Virgili
Zitieren
APA
Blanco-Justicia, A., Jebreel, N., Manzanares-Salor, B., Sánchez, D., Domingo-Ferrer, J., Collell, G., & Tan, K. E. (2025). Digital forgetting in large language models: a survey of unlearning methods. Artificial Intelligence Review, 58(3). https://doi.org/10.1007/s10462-024-11078-6
Vancouver
Blanco-Justicia A, Jebreel N, Manzanares-Salor B, Sánchez D, Domingo-Ferrer J, Collell G, et al. Digital forgetting in large language models: a survey of unlearning methods. Artificial Intelligence Review. 2025;58(3). doi:10.1007/s10462-024-11078-6
BibTeX
@article{BlancoJusticia2025,
author = {Blanco-Justicia, Alberto and Jebreel, Najeeb and Manzanares-Salor, Benet and Sánchez, David and Domingo-Ferrer, Josep and Collell, Guillem and Tan, Kuan Eeik},
title = {Digital forgetting in large language models: a survey of unlearning methods},
journal = {Artificial Intelligence Review},
year = {2025},
volume = {58},
number = {3},
doi = {10.1007/s10462-024-11078-6}
}
EUVIMED