EUVIMEDEUVIMED entdecken
Crossref

GPU-Accelerated Edge Inference for COVID-19 Chest X-Ray Classification: A Systematic Benchmarking of CNN Architectures on the NVIDIA Jetson Orin Nano with TensorRT FP16 and INT8 Optimisation

Bharat Tank, Mitul Patel, Soumya Das

Dandao Xuebao/Journal of Ballistics · 31.12.2025 · 37 (1) · 466-489

DOI: 10.52783/dxjb.v37.363 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Dandao Xuebao/Journal of Ballistics, online erschienen am 31.12.2025. Verfasst von Bharat Tank und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Convolutional neural networks (CNNs) have demonstrated strong diagnostic performance for automated COVID-19 classification from chest X-ray (CXR) images; nonetheless, published studies evaluate inference almost exclusively on server-grade GPU hardware that are economically and logistically incompatible with point-of-care deployment in resource-constrained clinical settings. GPU-accelerated embedded devices offer a viable alternative deployment pathway, although a systematic characterisation of CNN inference performance, power consumption, and diagnostic accuracy across multiple precision modes on these devices is still not present in the medical imaging literature.

Methods

This study presents a systematic benchmarking evaluation of four representative CNN architectures — ResNet18, ResNet50, DenseNet121, and SqueezeNet deployed on the NVIDIA Jetson Orin Nano (1024-core Ampere GPU, 32 Tensor Cores, 8 GB LPDDR5 unified memory, less than USD-200) in binary classification of COVID-19 CXR images using the COVID-Xray-5k dataset (5,000 images). The diagnostic performance was assessed via 5-fold stratified cross-validation and tested on an independent 918-image held-out test partition. All architectures were characterised in three precision modes, namely FP32 baseline, TensorRT FP16 half-precision, and INT8 post-training quantisation (PTQ). Inference latency (mean, P50, P95), throughput (FPS), peak GPU memory utilisation, and power consumption of the board were system-level profiled in MAXN mode via sudo tegrastats at 100 ms polling rates.

Results

AUC ≥ 0.978 All four architectures achieved AUC ≥ 0.978, and diagnostic accuracy was fully preserved under FP16 and INT8 TRT (ΔAUC < 0.005). FP32 latency, FP16 TRT latency and INT8 TRT latency were 15.7 ms (63.8 FPS, 7.4 W, 8.6 inferences/Joule), 10.8 ms (92.0 FPS, 7.4 W, 12.4 inferences/Joule) and 7.2 ms (138.9 FPS, 6.9 W, 20.1 inferences/Joule) on ResNet18 respectively. The highest FPS of SqueezeNet was 909 FPS gained under INT8 TRT at around 1.5 MB of GPU memory. FP32 GPU inference achieved 856–2,178× speedup over the Raspberry Pi 4 CPU baseline; TensorRT FP16 extended speedups to 1,244–5,290×. A novel empirical finding is that DenseNet121 has serialised GPU memory access, which causes it to have higher FP32 inference latency than ResNet50 despite having fewer parameters due to the dense connectivity.

Conclusions

TensorRT-optimised CNN inference on the NVIDIA Jetson Orin Nano enables real-time CXR screening within a sub-7.4 W average power. The reproducible three-precision-mode deployment pipeline, fully power-profiled architecture selection criteria presented here provide a strict empirical basis for deploying CNN-based diagnostics on GPU-accelerated embedded hardware in resource-constrained healthcare settings embedded hardware.

Zitieren

APA

Tank, B., Patel, M., & Das, S. (2025). GPU-Accelerated Edge Inference for COVID-19 Chest X-Ray Classification: A Systematic Benchmarking of CNN Architectures on the NVIDIA Jetson Orin Nano with TensorRT FP16 and INT8 Optimisation. Dandao Xuebao/Journal of Ballistics, 37(1), 466-489. https://doi.org/10.52783/dxjb.v37.363

Vancouver

Tank B, Patel M, Das S. GPU-Accelerated Edge Inference for COVID-19 Chest X-Ray Classification: A Systematic Benchmarking of CNN Architectures on the NVIDIA Jetson Orin Nano with TensorRT FP16 and INT8 Optimisation. Dandao Xuebao/Journal of Ballistics. 2025;37(1):466-489. doi:10.52783/dxjb.v37.363

BibTeX
@article{Tank2025,
  author = {Tank, Bharat and Patel, Mitul and Das, Soumya},
  title = {GPU-Accelerated Edge Inference for COVID-19 Chest X-Ray Classification: A Systematic Benchmarking of CNN Architectures on the NVIDIA Jetson Orin Nano with TensorRT FP16 and INT8 Optimisation},
  journal = {Dandao Xuebao/Journal of Ballistics},
  year = {2025},
  volume = {37},
  number = {1},
  pages = {466--489},
  doi = {10.52783/dxjb.v37.363}
}
Kostenlos starten →