OrCA: An Efficient Computing Architecture for Fault-Tolerant Large Language Model Inference via Outlier-Aware Recalculation
ACM Transactions on Design Automation of Electronic Systems · 30.11.2026 · 31 (6) · 1-37
DOI: 10.1145/3801976 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Design Automation of Electronic Systems, online erschienen am 13.06.2026. Verfasst von Yihao Shi und 7 weiteren Autor:innen. Die Arbeit nennt 82 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Deep-learning accelerators such as TPUs and GPUs now run ever larger models. Conventional fault-tolerant accelerators, designed for CNNs, are ineffective and cost-prohibitive for emerging large language models (LLMs) due to their exponentially higher computational demands. To address the impact of soft errors on computation, various specialized fault-tolerant DNN accelerators have been proposed, typically employing full-element protection. Yet, emerging LLMs exhibit exponentially higher computational demands compared to traditional CNN models, rendering conventional fault-tolerant accelerators both cost-prohibitive and ineffective in handling multi-point faults. To tackle these challenges, we conduct fault injection experiments on multiple representative LLMs, revealing the inherent parameter redundancy in Transformer-based models. Specifically, only 1%–2% of the elements significantly affect the output when perturbed—these critical elements are identified as outliers. Leveraging this insight, we propose OrCA, a hierarchically redundant fault-tolerant accelerator, which introduces the principle of selective protection for critical elements and optimizes the dataflow accordingly. Through extensive fault injection experiments and hardware simulations, we demonstrate that OrCA outperforms conventional fault-tolerant accelerators, achieving superior protection at equal or lower area overhead. Notably, OrCA delivers better performance under fault rates up to 10× higher and supports elastic protection against diverse hardware faults (e.g., transient and permanent faults), adapting to varying fault-tolerance requirements. Furthermore, OrCA breaks the limitation of traditional accelerators that require separate error detection and correction steps, enabling more efficient fault resilience.
Zitieren
APA
Shi, Y., Ma, S., Li, T., Wang, B., Luo, S., Zhao, Y., Zhang, J., & Wu, L. (2026). OrCA: An Efficient Computing Architecture for Fault-Tolerant Large Language Model Inference via Outlier-Aware Recalculation. ACM Transactions on Design Automation of Electronic Systems, 31(6), 1-37. https://doi.org/10.1145/3801976
Vancouver
Shi Y, Ma S, Li T, Wang B, Luo S, Zhao Y, et al. OrCA: An Efficient Computing Architecture for Fault-Tolerant Large Language Model Inference via Outlier-Aware Recalculation. ACM Transactions on Design Automation of Electronic Systems. 2026;31(6):1-37. doi:10.1145/3801976
BibTeX
@article{Shi2026,
author = {Shi, Yihao and Ma, Sheng and Li, Tiejun and Wang, Bo and Luo, Shengbai and Zhao, Yun and Zhang, Jianming and Wu, Lizhou},
title = {OrCA: An Efficient Computing Architecture for Fault-Tolerant Large Language Model Inference via Outlier-Aware Recalculation},
journal = {ACM Transactions on Design Automation of Electronic Systems},
year = {2026},
volume = {31},
number = {6},
pages = {1--37},
doi = {10.1145/3801976}
}
EUVIMED