EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

TRIDENT: Efficient Small-Large Model Collaboration via Heterogeneous Expert Decoupling

Guangyu Dai, Siliang Tang, Yueting Zhuang

Electronics · 17.04.2026 · 15 (8) · 1699

DOI: 10.3390/electronics15081699 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 17.04.2026. Verfasst von Guangyu Dai und 2 weiteren Autor:innen. Die Arbeit nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The burgeoning scale of Pre-trained Large Models (PLMs) has intensified the demand for efficient inference without compromising performance, while existing large model collaborative frameworks have shown promise, they often suffer from functional redundancy among experts and limited robustness in complex cross-domain scenarios. In this paper, we propose Tri-gate Routing for Inference via Decoupled Efficient Network Technologies (TRIDENT), a highly efficient and robust heterogeneous collaborative inference framework. TRIDENT leverages the complementary inductive biases of MLP (for statistical patterns) and KAN (for symbolic logic) to maximize reasoning potential with minimal parametric overhead. To address feature homogenization in traditional distillation, we introduce Orthogonal Feature Decoupling Distillation, utilizing an orthogonality loss Lorth for functional decoupling and a reconstruction loss Lrecon to anchor decoupled features to the PLM knowledge manifold. During inference, a Dual-Threshold Arbiter effectively detects expert hallucinations by integrating individual confidence τcon and heterogeneous consistency τagree. Extensive experiments on CIFAR-100-LT, XNLI, and GSM8K demonstrate that TRIDENT significantly reduces the Invocation Rate (IR) of PLMs while maintaining high accuracy. Our findings reveal a distinct Pareto optimal balance and validate the spontaneous division of labor between heterogeneous experts. By transcending the limitations of single-architecture systems, TRIDENT provides a robust and interpretable pathway for efficient collaborative intelligence.

Zitieren

APA

Dai, G., Tang, S., & Zhuang, Y. (2026). TRIDENT: Efficient Small-Large Model Collaboration via Heterogeneous Expert Decoupling. Electronics, 15(8), 1699. https://doi.org/10.3390/electronics15081699

Vancouver

Dai G, Tang S, Zhuang Y. TRIDENT: Efficient Small-Large Model Collaboration via Heterogeneous Expert Decoupling. Electronics. 2026;15(8):1699. doi:10.3390/electronics15081699

BibTeX
@article{Dai2026,
  author = {Dai, Guangyu and Tang, Siliang and Zhuang, Yueting},
  title = {TRIDENT: Efficient Small-Large Model Collaboration via Heterogeneous Expert Decoupling},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {8},
  pages = {1699},
  doi = {10.3390/electronics15081699}
}
Kostenlos starten →