EUVIMEDEUVIMED entdecken
Crossref

On the Feasibility Boundaries of LLM Prefill Inference on Unified-Memory Edge SoCs

Juyeon Lee, Jiayue Wang, Soojung Ryu, Hyuk-Jae Lee

IEEE Computer Architecture Letters · 2026-07 · 25 (2) · 267-270

DOI: 10.1109/lca.2026.3703423 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Computer Architecture Letters, 2026-07. Verfasst von Juyeon Lee und 3 weiteren Autor:innen. Die Arbeit nennt 15 Literaturangaben. Gefördert von: Institute of Information & Communications Technology Planning and Evaluation, Korea government.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

LLM inference on unified-memory (UM) edge SoCs is commonly assumed to follow a GPU-centric execution model once nominal memory capacity appears sufficient. Our experiments on an NVIDIA Jetson Orin Nano 8 GB reveal a prefill feasibility boundary in the evaluated modified FlexGen-based runtime, where execution stability becomes sensitive to runtime memory-management behavior. Under GPU-centric execution, collapse is first observed at batch size 8 with zram enabled, and the boundary shifts to batch size 16 under a modified memory configuration. At these boundary points, completion becomes probabilistic, with only 23%–33% of runs succeeding; all failures occur before decode. Notably, collapse can occur despite apparent physical-memory headroom, showing that nominal capacity alone does not determine execution stability. These findings show that, in the evaluated setup, prefill completion acts as an effective feasibility gate: latency and throughput become meaningful only after execution feasibility is established.

Förderung

  • Institute of Information & Communications Technology Planning and Evaluation
  • Korea government

Zitieren

APA

Lee, J., Wang, J., Ryu, S., & Lee, H. (2026). On the Feasibility Boundaries of LLM Prefill Inference on Unified-Memory Edge SoCs. IEEE Computer Architecture Letters, 25(2), 267-270. https://doi.org/10.1109/lca.2026.3703423

Vancouver

Lee J, Wang J, Ryu S, Lee H. On the Feasibility Boundaries of LLM Prefill Inference on Unified-Memory Edge SoCs. IEEE Computer Architecture Letters. 2026;25(2):267-270. doi:10.1109/lca.2026.3703423

BibTeX
@article{Lee2026,
  author = {Lee, Juyeon and Wang, Jiayue and Ryu, Soojung and Lee, Hyuk-Jae},
  title = {On the Feasibility Boundaries of LLM Prefill Inference on Unified-Memory Edge SoCs},
  journal = {IEEE Computer Architecture Letters},
  year = {2026},
  volume = {25},
  number = {2},
  pages = {267--270},
  doi = {10.1109/lca.2026.3703423}
}
Kostenlos starten →