On the Feasibility Boundaries of LLM Prefill Inference on Unified-Memory Edge SoCs
IEEE Computer Architecture Letters · 2026-07 · 25 (2) · 267-270
DOI: 10.1109/lca.2026.3703423 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Computer Architecture Letters, 2026-07. Verfasst von Juyeon Lee und 3 weiteren Autor:innen. Die Arbeit nennt 15 Literaturangaben. Gefördert von: Institute of Information & Communications Technology Planning and Evaluation, Korea government.
Abstract (Originalsprache)
LLM inference on unified-memory (UM) edge SoCs is commonly assumed to follow a GPU-centric execution model once nominal memory capacity appears sufficient. Our experiments on an NVIDIA Jetson Orin Nano 8 GB reveal a prefill feasibility boundary in the evaluated modified FlexGen-based runtime, where execution stability becomes sensitive to runtime memory-management behavior. Under GPU-centric execution, collapse is first observed at batch size 8 with zram enabled, and the boundary shifts to batch size 16 under a modified memory configuration. At these boundary points, completion becomes probabilistic, with only 23%–33% of runs succeeding; all failures occur before decode. Notably, collapse can occur despite apparent physical-memory headroom, showing that nominal capacity alone does not determine execution stability. These findings show that, in the evaluated setup, prefill completion acts as an effective feasibility gate: latency and throughput become meaningful only after execution feasibility is established.
Förderung
- Institute of Information & Communications Technology Planning and Evaluation
- Korea government
Zitieren
APA
Lee, J., Wang, J., Ryu, S., & Lee, H. (2026). On the Feasibility Boundaries of LLM Prefill Inference on Unified-Memory Edge SoCs. IEEE Computer Architecture Letters, 25(2), 267-270. https://doi.org/10.1109/lca.2026.3703423
Vancouver
Lee J, Wang J, Ryu S, Lee H. On the Feasibility Boundaries of LLM Prefill Inference on Unified-Memory Edge SoCs. IEEE Computer Architecture Letters. 2026;25(2):267-270. doi:10.1109/lca.2026.3703423
BibTeX
@article{Lee2026,
author = {Lee, Juyeon and Wang, Jiayue and Ryu, Soojung and Lee, Hyuk-Jae},
title = {On the Feasibility Boundaries of LLM Prefill Inference on Unified-Memory Edge SoCs},
journal = {IEEE Computer Architecture Letters},
year = {2026},
volume = {25},
number = {2},
pages = {267--270},
doi = {10.1109/lca.2026.3703423}
}
EUVIMED