EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Accelerating Deep Learning Inference: A Comparative Analysis of Modern Acceleration Frameworks

Ishrak Jahan Ratul, Yuxiao Zhou, Kecheng Yang

Electronics · 25.07.2025 · 14 (15) · 2977

DOI: 10.3390/electronics14152977 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 25.07.2025. Verfasst von Ishrak Jahan Ratul und 2 weiteren Autor:innen. Die Arbeit wurde bisher 31-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NSF.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deep learning (DL) continues to play a pivotal role in a wide range of intelligent systems, including autonomous machines, smart surveillance, industrial automation, and portable healthcare technologies. These applications often demand low-latency inference and efficient resource utilization, especially when deployed on embedded or edge devices with limited computational capacity. As DL models become increasingly complex, selecting the right inference framework is essential to meeting performance and deployment goals. In this work, we conduct a comprehensive comparison of five widely adopted inference frameworks: PyTorch, ONNX Runtime, TensorRT, Apache TVM, and JAX. All experiments are performed on the NVIDIA Jetson AGX Orin platform, a high-performance computing solution tailored for edge artificial intelligence workloads. The evaluation considers several key performance metrics, including inference accuracy, inference time, throughput, memory usage, and power consumption. Each framework is tested using a wide range of convolutional and transformer models and analyzed in terms of deployment complexity, runtime efficiency, and hardware utilization. Our results show that certain frameworks offer superior inference speed and throughput, while others provide advantages in flexibility, portability, or ease of integration. We also observe meaningful differences in how each framework manages system memory and power under various load conditions. This study offers practical insights into the trade-offs associated with deploying DL inference on resource-constrained hardware.

Förderung

  • NSF

Zitieren

APA

Ratul, I. J., Zhou, Y., & Yang, K. (2025). Accelerating Deep Learning Inference: A Comparative Analysis of Modern Acceleration Frameworks. Electronics, 14(15), 2977. https://doi.org/10.3390/electronics14152977

Vancouver

Ratul IJ, Zhou Y, Yang K. Accelerating Deep Learning Inference: A Comparative Analysis of Modern Acceleration Frameworks. Electronics. 2025;14(15):2977. doi:10.3390/electronics14152977

BibTeX
@article{Ratul2025,
  author = {Ratul, Ishrak Jahan and Zhou, Yuxiao and Yang, Kecheng},
  title = {Accelerating Deep Learning Inference: A Comparative Analysis of Modern Acceleration Frameworks},
  journal = {Electronics},
  year = {2025},
  volume = {14},
  number = {15},
  pages = {2977},
  doi = {10.3390/electronics14152977}
}
Kostenlos starten →