EUVIMEDEUVIMED entdecken
Crossref

Computational Cost and Output Quality Analysis of Prompt Engineering Methods in AI Systems

Kasimala

Journal of Information Systems Engineering & Management · 30.12.2024

DOI: 10.52783/jisem.v9i4.166 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Information Systems Engineering & Management, online erschienen am 30.12.2024. Verfasst von Kasimala.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The fast development of Large Language Models (LLMs) has triggered a paradigm shift in the fine-tuning-based model-specific approach to in-context learning via prompt engineering. The research paper presents a synthesis of computational cost and quality output of different prompt engineering methods that have been developed until 2024 in a systematized way. The trade-offs between inferential accuracy and resource expenditure are measured by conducting a rigorous study of techniques including zero-shot, few-shot, and Chain-of-Thought (CoT) prompting. Results of the canonical literature suggest that CoT prompting can improve reasoning by over 20-30 percent on complex benchmarks, such as GSM8K, at the same time similar platforms like CoT prompting induce a two to fivefold increase in token use and delay. New optimization techniques such as a model cascading, prompt compression and batch prompting employed by FrugalGPT are explored as being capable of reducing these costs. In particular, FrugalGPT is demonstrated to save up to 98 percent of the API expenses and outperform or even surpass the standalone high-tier fashions. Another tool of measurement discussed in this paper is the Economical Prompting Index (EPI) which is a standardized measure of assessing the efficiency of prompt strategies. The results highlight the necessity of balanced architectural designs that could focus on reasoning fidelity and financial viability in AI implementations in enterprises. Among the most important ones, there is the internalization of recurrent prompts which reduces up to 25% inference costs and the influence of output length on the reasoning efficiency.

Zitieren

APA

Kasimala (2024). Computational Cost and Output Quality Analysis of Prompt Engineering Methods in AI Systems. Journal of Information Systems Engineering & Management. https://doi.org/10.52783/jisem.v9i4.166

Vancouver

Kasimala. Computational Cost and Output Quality Analysis of Prompt Engineering Methods in AI Systems. Journal of Information Systems Engineering & Management. 2024. doi:10.52783/jisem.v9i4.166

BibTeX
@article{Kasimala2024,
  author = {Kasimala},
  title = {Computational Cost and Output Quality Analysis of Prompt Engineering Methods in AI Systems},
  journal = {Journal of Information Systems Engineering & Management},
  year = {2024},
  doi = {10.52783/jisem.v9i4.166}
}
Kostenlos starten →