Using edge computing and Large Language Models (LLM) for constructing a local agent
JURNAL ELTEK · 30.10.2026 · 24 (2) · 131-144
DOI: 10.33795/eltek.v24i2.11288 ↗Kurz gefasst
Fachartikel in der Zeitschrift JURNAL ELTEK, online erschienen am 30.10.2026. Verfasst von Setya Widyawan Prakosa und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).
Abstract (Originalsprache)
The growing demand for artificial intelligence systems that are responsive, privacy-preserving, and capable of operating autonomously is driving the adoption of Large Language Models (LLMs) on edge devices. However, deploying LLMs as the core of agents on devices with limited resources still faces computational and memory constraints. This research develops an LLM-based local agent architecture that runs locally on a Raspberry Pi without relying on cloud services. The system integrates local LLM inference with agentic functions, including intent processing, reasoning, tool invocation, and interaction with hardware via GPIO. Several quantized Small Language Models (SLMs) were evaluated on a Raspberry Pi 5 with 16 GB of RAM using CPU-based inference. The evaluation included total latency, Time to First Token (TTFT), token generation rate, peak RAM usage, peak CPU utilization, and peak temperature. The results showed significant performance differences among the models. Qwen 2.5 0.5B Q3 produced the lowest total latency of 23.9 seconds, the lowest TTFT of 269.0 ms, and the highest generation rate of 12.1 tokens/s. However, the results indicate that inference speed alone is not sufficient to determine the most suitable model as the core of a local agent. Most models reached peak CPU utilization above 95%, while RAM usage and thermal characteristics varied across configurations. Based on a combination of inference performance and resource usage, models in the 1B–1.7B range specifically LLaMA 3.2 1B Q4, Qwen 2.5 1.5B Q4, and SmolLM 1.7B Q4 show potential as a balanced set of candidates between responsiveness and model capacity. The research results demonstrate the feasibility of deploying LLMs as local agents on low-cost edge devices and provide a basis for selecting models based on a balance between inference efficiency, resource constraints, and agentic capabilities.
Zitieren
APA
Prakosa, S. W., Rizal, N. A., & Kurniati, T. (2026). Using edge computing and Large Language Models (LLM) for constructing a local agent. JURNAL ELTEK, 24(2), 131-144. https://doi.org/10.33795/eltek.v24i2.11288
Vancouver
Prakosa SW, Rizal NA, Kurniati T. Using edge computing and Large Language Models (LLM) for constructing a local agent. JURNAL ELTEK. 2026;24(2):131-144. doi:10.33795/eltek.v24i2.11288
BibTeX
@article{Prakosa2026,
author = {Prakosa, Setya Widyawan and Rizal, Naufal Ainur and Kurniati, Tias},
title = {Using edge computing and Large Language Models (LLM) for constructing a local agent},
journal = {JURNAL ELTEK},
year = {2026},
volume = {24},
number = {2},
pages = {131--144},
doi = {10.33795/eltek.v24i2.11288}
}
EUVIMED