EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Fast GPU Inference With Unstructured-Pruned Sparse DNN Architectures

Masahiko Ando, Keita Yamane, Takashi Oshima

IEEE Access · 2026 · 14 · 68271-68280

DOI: 10.1109/access.2026.3686628 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Masahiko Ando und 2 weiteren Autor:innen. Die Arbeit nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: “Research and Development of Enhanced Infrastructures for Post 5G Information Communication Systems” commissioned by the New Energy and Industrial Technology D….

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Achieving fast and accurate deep neural network (DNN) inference on edge devices requires both aggressive model compression and efficient execution pipelines. This work introduces a sparsity-aware compilation framework that integrates unstructured pruning with a hybrid SparseRT/TensorRT optimization flow. Using SynFlow-based pruning at initialization, we produce models with over 99.9% sparsity while avoiding layer collapse. The resulting sparse networks are compiled into TensorRT engines through custom SparseRT kernels, fully eliminating CPU–GPU data-transfer overhead and enabling end-to-end sparse execution. Applied to explainable deep one-class classification (XDOC) for anomaly detection on MVTec AD dataset, our method achieves sub-1-ms inference on edge GPUs while preserving over 90% AUC accuracy. Replacing VGG11-BN with RepVGG and incorporating CutPaste-augmented learning further increases robustness under extreme pruning, achieving a parameter budget of 0.4% with a 0.98-ms inference time. These results demonstrate the effectiveness of combining unstructured pruning, architecture selection, and compiler-level integration for ultra-low-latency edge-AI inference in post-5G digital-twin applications.

Förderung

  • “Research and Development of Enhanced Infrastructures for Post 5G Information Communication Systems” commissioned by the New Energy and Industrial Technology D…

Zitieren

APA

Ando, M., Yamane, K., & Oshima, T. (2026). Fast GPU Inference With Unstructured-Pruned Sparse DNN Architectures. IEEE Access, 14, 68271-68280. https://doi.org/10.1109/access.2026.3686628

Vancouver

Ando M, Yamane K, Oshima T. Fast GPU Inference With Unstructured-Pruned Sparse DNN Architectures. IEEE Access. 2026;14:68271-68280. doi:10.1109/access.2026.3686628

BibTeX
@article{Ando2026,
  author = {Ando, Masahiko and Yamane, Keita and Oshima, Takashi},
  title = {Fast GPU Inference With Unstructured-Pruned Sparse DNN Architectures},
  journal = {IEEE Access},
  year = {2026},
  volume = {14},
  pages = {68271--68280},
  doi = {10.1109/access.2026.3686628}
}
Kostenlos starten →