MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (17) · 19449-19457
DOI: 10.1609/aaai.v38i17.29916 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Lang Yu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 26-mal zitiert (Crossref).
Abstract (Originalsprache)
Large language models (LLMs) have shown great success in various Natural Language Processing (NLP) tasks, whist they still need updates after deployment to fix errors or keep pace with the changing knowledge in the world. Researchers formulate such problem as Model Editing and have developed various editors focusing on different axes of editing properties. However, current editors can hardly support all properties and rely on heavy computational resources. In this paper, we propose a plug-in Model Editing method based on neuron-indexed dynamic LoRA (MELO), which alters the behavior of language models by dynamically activating certain LoRA blocks according to the index built in an inner vector database. Our method satisfies various editing properties with high efficiency and can be easily integrated into multiple LLM backbones. Experimental results show that our proposed MELO achieves state-of-the-art editing performance on three sequential editing tasks (document classification, question answering and hallucination correction), while requires the least trainable parameters and computational cost.
Zitieren
APA
Yu, L., Chen, Q., Zhou, J., & He, L. (2024). MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA. Proceedings of the AAAI Conference on Artificial Intelligence, 38(17), 19449-19457. https://doi.org/10.1609/aaai.v38i17.29916
Vancouver
Yu L, Chen Q, Zhou J, He L. MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(17):19449-19457. doi:10.1609/aaai.v38i17.29916
BibTeX
@article{Yu2024,
author = {Yu, Lang and Chen, Qin and Zhou, Jie and He, Liang},
title = {MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {17},
pages = {19449--19457},
doi = {10.1609/aaai.v38i17.29916}
}
EUVIMED