EUVIMEDEUVIMED entdecken
Crossref

Multiplicative Orthogonal Sequential Editing for Language Models

Hao-Xiang Xu, Jun-Yu Ma, Ziqi Peng, Yuhao Sun, Zhen-Hua Ling, Jia-Chen Gu

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (40) · 34124-34132

DOI: 10.1609/aaai.v40i40.40707 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Hao-Xiang Xu und 5 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Knowledge editing aims to efficiently modify the internal knowledge of large language models (LLMs) without compromising their other capabilities. The prevailing editing paradigm, which appends an update matrix to the original parameter matrix, has been shown by some studies to damage key numerical stability indicators (such as condition number and norm), thereby reducing editing performance and general abilities, especially in sequential editing scenario. Although subsequent methods have made some improvements, they remain within the additive framework and have not fundamentally addressed this limitation. To solve this problem, we analyze it from both statistical and mathematical perspectives and conclude that multiplying the original matrix by an orthogonal matrix does not change the numerical stability of the matrix. Inspired by this, different from the previous additive editing paradigm, a multiplicative editing paradigm termed Multiplicative Orthogonal Sequential Editing (MOSE) is proposed. Specifically, we first derive the matrix update in the multiplicative form, the new knowledge is then incorporated into an orthogonal matrix, which is multiplied by the original parameter matrix. In this way, the numerical stability of the edited matrix is unchanged, thereby maintaining editing performance and general abilities. We compared MOSE with several current knowledge editing methods, systematically evaluating their impact on both editing performance and the general abilities across three different LLMs. Experimental results show that MOSE effectively limits deviations in the edited parameter matrix and maintains its numerical stability. Compared to current methods, MOSE achieves a 12.08% improvement in sequential editing performance, while retaining 95.73% of general abilities across downstream tasks.

Zitieren

APA

Xu, H., Ma, J., Peng, Z., Sun, Y., Ling, Z., & Gu, J. (2026). Multiplicative Orthogonal Sequential Editing for Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(40), 34124-34132. https://doi.org/10.1609/aaai.v40i40.40707

Vancouver

Xu H, Ma J, Peng Z, Sun Y, Ling Z, Gu J. Multiplicative Orthogonal Sequential Editing for Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(40):34124-34132. doi:10.1609/aaai.v40i40.40707

BibTeX
@article{Xu2026,
  author = {Xu, Hao-Xiang and Ma, Jun-Yu and Peng, Ziqi and Sun, Yuhao and Ling, Zhen-Hua and Gu, Jia-Chen},
  title = {Multiplicative Orthogonal Sequential Editing for Language Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {40},
  pages = {34124--34132},
  doi = {10.1609/aaai.v40i40.40707}
}
Kostenlos starten →