EUVIMEDEUVIMED entdecken
StudienprotokollOpen Access · CC BY 4.0

Protocol and Software Co-optimization: Overcoming the PCIe Transfer Bottleneck in dGPU Accelerated In-kernel ML

WenTao Li, SongQuan Zhu, Panpan Cuan, Zheng Zhang, Jie Zhao, Qinglei Qi

ACM Transactions on Architecture and Code Optimization · 30.09.2026 · 23 (3) · 1-26

DOI: 10.1145/3821578 ↗

Kurz gefasst

Studienprotokoll in der Zeitschrift ACM Transactions on Architecture and Code Optimization, online erschienen am 24.07.2026. Verfasst von WenTao Li und 5 weiteren Autor:innen. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Key Research and Development and Promotion Program of Henan Province, China, Science and Technology Program of Nanyang City, China, Doctoral Research Foundation of Nanyang Normal University, China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In-kernel machine learning (in-kernel ML) enables low-overhead online decision-making for OS subsystems such as scheduling, storage, and security. However, in the current in-kernel-to-dGPU offloading path, inference data must traverse kernel space, user space, and device memory, making the software path and PCIe transfers the dominant costs in the small-batch regime and often erasing acceleration gains, rather than being limited by insufficient dGPU compute capability. To address this issue, this article proposes a hierarchical co-optimization scheme of data reduction, lightweight compression, and protocol optimization: it combines feature selection, mixed-precision quantization, and lightweight compression to reduce transfer load, and uses a state-aware dynamic scheduler to adapt the transfer strategy to changes in PCIe bandwidth, CPU load, and data sparsity. On the dGPU, decompression and dequantization are pipelined with computation to increase overlap. Across multiple in-kernel workloads, our scheme reduces transferred bytes by 50%–87.5%, lowers the dGPU break-even batch size from 256 to 64, reduces end-to-end inference latency by 3.2×–5.8×, and keeps the accuracy loss within 3%.

Förderung

  • Key Research and Development and Promotion Program of Henan Province, China
  • Science and Technology Program of Nanyang City, China
  • Doctoral Research Foundation of Nanyang Normal University, China
  • Joint Fund for Science and Technology Research and Development of Henan Province, China

Zitieren

APA

Li, W., Zhu, S., Cuan, P., Zhang, Z., Zhao, J., & Qi, Q. (2026). Protocol and Software Co-optimization: Overcoming the PCIe Transfer Bottleneck in dGPU Accelerated In-kernel ML. ACM Transactions on Architecture and Code Optimization, 23(3), 1-26. https://doi.org/10.1145/3821578

Vancouver

Li W, Zhu S, Cuan P, Zhang Z, Zhao J, Qi Q. Protocol and Software Co-optimization: Overcoming the PCIe Transfer Bottleneck in dGPU Accelerated In-kernel ML. ACM Transactions on Architecture and Code Optimization. 2026;23(3):1-26. doi:10.1145/3821578

BibTeX
@article{Li2026,
  author = {Li, WenTao and Zhu, SongQuan and Cuan, Panpan and Zhang, Zheng and Zhao, Jie and Qi, Qinglei},
  title = {Protocol and Software Co-optimization: Overcoming the PCIe Transfer Bottleneck in dGPU Accelerated In-kernel ML},
  journal = {ACM Transactions on Architecture and Code Optimization},
  year = {2026},
  volume = {23},
  number = {3},
  pages = {1--26},
  doi = {10.1145/3821578}
}
Kostenlos starten →