EUVIMEDObjevte EUVIMED
Open access · CC BY 4.0

OrCA: An Efficient Computing Architecture for Fault-Tolerant Large Language Model Inference via Outlier-Aware Recalculation

Yihao Shi, Sheng Ma, Tiejun Li, Bo Wang, Shengbai Luo, Yun Zhao, Jianming Zhang, Lizhou Wu

ACM Transactions on Design Automation of Electronic Systems · 2026-11-30 · 31 (6) · 1-37

DOI: 10.1145/3801976 ↗

At a glance

Journal article in ACM Transactions on Design Automation of Electronic Systems, published online on 2026-06-13. Written by Yihao Shi and 7 co-authors. 82 references. The full text is freely available (open access, CC BY 4.0).

AbstraktBibliografické údajeCiteRelated studies

Abstract (original language)

Deep-learning accelerators such as TPUs and GPUs now run ever larger models. Conventional fault-tolerant accelerators, designed for CNNs, are ineffective and cost-prohibitive for emerging large language models (LLMs) due to their exponentially higher computational demands. To address the impact of soft errors on computation, various specialized fault-tolerant DNN accelerators have been proposed, typically employing full-element protection. Yet, emerging LLMs exhibit exponentially higher computational demands compared to traditional CNN models, rendering conventional fault-tolerant accelerators both cost-prohibitive and ineffective in handling multi-point faults. To tackle these challenges, we conduct fault injection experiments on multiple representative LLMs, revealing the inherent parameter redundancy in Transformer-based models. Specifically, only 1%–2% of the elements significantly affect the output when perturbed—these critical elements are identified as outliers. Leveraging this insight, we propose OrCA, a hierarchically redundant fault-tolerant accelerator, which introduces the principle of selective protection for critical elements and optimizes the dataflow accordingly. Through extensive fault injection experiments and hardware simulations, we demonstrate that OrCA outperforms conventional fault-tolerant accelerators, achieving superior protection at equal or lower area overhead. Notably, OrCA delivers better performance under fault rates up to 10× higher and supports elastic protection against diverse hardware faults (e.g., transient and permanent faults), adapting to varying fault-tolerance requirements. Furthermore, OrCA breaks the limitation of traditional accelerators that require separate error detection and correction steps, enabling more efficient fault resilience.

Cite

APA

Shi, Y., Ma, S., Li, T., Wang, B., Luo, S., Zhao, Y., Zhang, J., & Wu, L. (2026). OrCA: An Efficient Computing Architecture for Fault-Tolerant Large Language Model Inference via Outlier-Aware Recalculation. ACM Transactions on Design Automation of Electronic Systems, 31(6), 1-37. https://doi.org/10.1145/3801976

Vancouver

Shi Y, Ma S, Li T, Wang B, Luo S, Zhao Y, et al. OrCA: An Efficient Computing Architecture for Fault-Tolerant Large Language Model Inference via Outlier-Aware Recalculation. ACM Transactions on Design Automation of Electronic Systems. 2026;31(6):1-37. doi:10.1145/3801976

BibTeX
@article{Shi2026,
  author = {Shi, Yihao and Ma, Sheng and Li, Tiejun and Wang, Bo and Luo, Shengbai and Zhao, Yun and Zhang, Jianming and Wu, Lizhou},
  title = {OrCA: An Efficient Computing Architecture for Fault-Tolerant Large Language Model Inference via Outlier-Aware Recalculation},
  journal = {ACM Transactions on Design Automation of Electronic Systems},
  year = {2026},
  volume = {31},
  number = {6},
  pages = {1--37},
  doi = {10.1145/3801976}
}
Start for free →