EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Evolutionary-Algorithm-Based Automatic Prompt Generation for Vision-Language Model Evaluation

Wang Li, Mei Hui, Wang Xiaolu, Li Shujuan

IEEE Access · 2026 · 14 · 875-885

DOI: 10.1109/access.2025.3648351 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Wang Li und 3 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Selected Projects by the Third Research Institute of the Ministry of Public Security, National Key Research and Development Program of China, Key Project of the Special Fund for Basic Scientific Research Business Expenses by the Third Research Institute of the Ministry of Public Security, in 2025.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper proposes an evolutionary algorithm-based automatic generation method for evaluation prompts. By constructing a structured prompt encoding model, multi-dimensional fitness function design, and population evolution mechanism, it realizes the automatic synthesis of high-quality prompts for Vision-Language Model (VLM) evaluation tasks. Firstly, this paper models the prompt structure as a set of combinable language units, covering core semantic components such as objects, actions, space, quantity, and context. Secondly, it designs a five-dimensional fitness evaluation index (semantic relevance, detail coverage, spatial relationship consistency, quantity matching, and robustness) to guide the evolution direction of the prompt population. Finally, it constructs a complete prompt evolution strategy by integrating mechanisms such as elitism preservation, semantic mutation, and structural crossover. Experiments are conducted on multiple mainstream vision-language models such as BLIP-2, MiniGPT-4, and LLaVA, covering three types of tasks: image-text generation, image-text matching, and open-ended question answering. The results show that compared with manual prompts, template prompts, and prompts generated by large language models, the prompts generated by this method have significant advantages in multiple evaluation indicators, especially in spatial reasoning, quantity understanding, and context restoration. Further ablation experiments and mechanism comparison analysis also verify the effectiveness of the evolutionary algorithm in optimizing prompt structure and improving evaluation quality.

Förderung

  • Selected Projects by the Third Research Institute of the Ministry of Public Security
  • National Key Research and Development Program of China
  • Key Project of the Special Fund for Basic Scientific Research Business Expenses by the Third Research Institute of the Ministry of Public Security, in 2025
  • Special Youth Fund for Basic Research Business Expenses by the Third Research Institute of the Ministry of Public Security, in 2025

Zitieren

APA

Li, W., Hui, M., Xiaolu, W., & Shujuan, L. (2026). Evolutionary-Algorithm-Based Automatic Prompt Generation for Vision-Language Model Evaluation. IEEE Access, 14, 875-885. https://doi.org/10.1109/access.2025.3648351

Vancouver

Li W, Hui M, Xiaolu W, Shujuan L. Evolutionary-Algorithm-Based Automatic Prompt Generation for Vision-Language Model Evaluation. IEEE Access. 2026;14:875-885. doi:10.1109/access.2025.3648351

BibTeX
@article{Li2026,
  author = {Li, Wang and Hui, Mei and Xiaolu, Wang and Shujuan, Li},
  title = {Evolutionary-Algorithm-Based Automatic Prompt Generation for Vision-Language Model Evaluation},
  journal = {IEEE Access},
  year = {2026},
  volume = {14},
  pages = {875--885},
  doi = {10.1109/access.2025.3648351}
}
Kostenlos starten →