EUVIMEDEUVIMED entdecken
Crossref

Outlier Matters: Efficient Long-to-Short Reasoning via Outlier-Guided Model Merging

Qiyuan Zhu, Dezhi Li, Lujun Li, Xiaoyu Qin, Wei Li, Hao Gu, Hua Xu, Sirui Han, Yike Guo

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (41) · 35213-35221

DOI: 10.1609/aaai.v40i41.40828 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Qiyuan Zhu und 8 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Reasoning Language Models (LRMs) have recently shown remarkable performance in complex reasoning tasks, but their extensive reasoning chains incur substantial computational overhead. To address this challenge, we propose Outlier-aware Reasoning Conciseness Adaptive Merge (ORCA), a novel plug-and-play model merging framework that leverages outlier activation patterns to fuse base models with reasoning models. Our ORCA introduces three key innovations: (1) adaptive alignment that reduces conflicts between disparate activation patterns during merging, (2) outlier-guided allocation that assigns merging coefficients proportional to each layer's reasoning importance as indicated by outlier concentrations, and (3) dynamic probe-based adjustment that adapts merging coefficients during inference based on input-specific activation characteristics. These strategies allow seamless integration into existing merging pipelines while creating unified models that maintain reasoning accuracy with significantly reduced response verbosity. Comprehensive evaluation across six benchmarks using Qwen and LLaMA models shows ORCA reduces average response length by 55% while improving accuracy by 2.4∼5.7% over existing methods.

Zitieren

APA

Zhu, Q., Li, D., Li, L., Qin, X., Li, W., Gu, H., Xu, H., Han, S., & Guo, Y. (2026). Outlier Matters: Efficient Long-to-Short Reasoning via Outlier-Guided Model Merging. Proceedings of the AAAI Conference on Artificial Intelligence, 40(41), 35213-35221. https://doi.org/10.1609/aaai.v40i41.40828

Vancouver

Zhu Q, Li D, Li L, Qin X, Li W, Gu H, et al. Outlier Matters: Efficient Long-to-Short Reasoning via Outlier-Guided Model Merging. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(41):35213-35221. doi:10.1609/aaai.v40i41.40828

BibTeX
@article{Zhu2026,
  author = {Zhu, Qiyuan and Li, Dezhi and Li, Lujun and Qin, Xiaoyu and Li, Wei and Gu, Hao and Xu, Hua and Han, Sirui and Guo, Yike},
  title = {Outlier Matters: Efficient Long-to-Short Reasoning via Outlier-Guided Model Merging},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {41},
  pages = {35213--35221},
  doi = {10.1609/aaai.v40i41.40828}
}
Kostenlos starten →