S HARING B EYOND D ECISION : Deep Collaboration between Large Language Models via Representation Ensemble
Transactions of the Association for Computational Linguistics · 17.07.2026 · 14 · 1765-1787
DOI: 10.1162/tacl.a.773 ↗Kurz gefasst
Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 17.07.2026. Verfasst von Yichong Huang und 9 weiteren Autor:innen. Die Arbeit nennt 66 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Large Language Models (LLMs) exhibit unique strengths arising from differences in model architecture, training data, and strategies. Ensemble learning has been explored to leverage these complementary strengths through decision-level sharing (i.e.,Decision Ensemble), which combines the predictions from multiple LLMs. However, such methods integrate only shallow decisions and overlook the exchange of deeper levels of information within the internal representations of LLMs, such as problem understanding, world knowledge, and latent reasoning patterns. In this work, we propose Representation Ensemble (RISE), a novel ensemble framework that enables cross-LLM representation sharing for richer information exchange. To address challenges of representation-level interaction caused by layer misalignment and latent-space incompatibility across LLMs, we introduce a representation alignment method based on relational similarity measures and an orthogonal latent-space transformation. Experimental results show that (1) RISE achieves performance competitive with existing decision ensemble methods, and (2) RISE is strongly complementary to decision ensemble, with their combination boosting collaboration gains by 14%–41%. Finally, we further compare ensemble of small LLMs to a single larger LLM and to model merging and composition approaches, and find that ensemble learning consistently generalizes well without additional training.
Zitieren
APA
Huang, Y., Feng, X., Fu, J., Feng, X., Li, B., Ye, Z., Qin, L., Fei, H., Ng, S., & Qin, B. (2026). S HARING B EYOND D ECISION : Deep Collaboration between Large Language Models via Representation Ensemble. Transactions of the Association for Computational Linguistics, 14, 1765-1787. https://doi.org/10.1162/tacl.a.773
Vancouver
Huang Y, Feng X, Fu J, Feng X, Li B, Ye Z, et al. S HARING B EYOND D ECISION : Deep Collaboration between Large Language Models via Representation Ensemble. Transactions of the Association for Computational Linguistics. 2026;14:1765-1787. doi:10.1162/tacl.a.773
BibTeX
@article{Huang2026,
author = {Huang, Yichong and Feng, Xiaocheng and Fu, Jinlan and Feng, Xiachong and Li, Baohang and Ye, Zekai and Qin, Libo and Fei, Hao and Ng, See-Kiong and Qin, Bing},
title = {S HARING B EYOND D ECISION : Deep Collaboration between Large Language Models via Representation Ensemble},
journal = {Transactions of the Association for Computational Linguistics},
year = {2026},
volume = {14},
pages = {1765--1787},
doi = {10.1162/tacl.a.773}
}
EUVIMED