EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Survey of Robot Intelligence with Large Language Models

Hyeongyo Jeong, Haechan Lee, Changwon Kim, Sungtae Shin

Applied Sciences · 02.10.2024 · 14 (19) · 8868

DOI: 10.3390/app14198868 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 02.10.2024. Verfasst von Hyeongyo Jeong und 3 weiteren Autor:innen. Die Arbeit wurde bisher 80-mal zitiert (Crossref) und nennt 231 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Trade, Industry, and Energy (MOTIE, Korea), Ministry of Education (MOE).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Since the emergence of ChatGPT, research on large language models (LLMs) has actively progressed across various fields. LLMs, pre-trained on vast text datasets, have exhibited exceptional abilities in understanding natural language and planning tasks. These abilities of LLMs are promising in robotics. In general, traditional supervised learning-based robot intelligence systems have a significant lack of adaptability to dynamically changing environments. However, LLMs help a robot intelligence system to improve its generalization ability in dynamic and complex real-world environments. Indeed, findings from ongoing robotics studies indicate that LLMs can significantly improve robots’ behavior planning and execution capabilities. Additionally, vision-language models (VLMs), trained on extensive visual and linguistic data for the vision question answering (VQA) problem, excel at integrating computer vision with natural language processing. VLMs can comprehend visual contexts and execute actions through natural language. They also provide descriptions of scenes in natural language. Several studies have explored the enhancement of robot intelligence using multimodal data, including object recognition and description by VLMs, along with the execution of language-driven commands integrated with visual information. This review paper thoroughly investigates how foundation models such as LLMs and VLMs have been employed to boost robot intelligence. For clarity, the research areas are categorized into five topics: reward design in reinforcement learning, low-level control, high-level planning, manipulation, and scene understanding. This review also summarizes studies that show how foundation models, such as the Eureka model for automating reward function design in reinforcement learning, RT-2 for integrating visual data, language, and robot actions in vision-language-action models, and AutoRT for generating feasible tasks and executing robot behavior policies via LLMs, have improved robot intelligence.

Förderung

  • Ministry of Trade, Industry, and Energy (MOTIE, Korea)
  • Ministry of Education (MOE)

Zitieren

APA

Jeong, H., Lee, H., Kim, C., & Shin, S. (2024). A Survey of Robot Intelligence with Large Language Models. Applied Sciences, 14(19), 8868. https://doi.org/10.3390/app14198868

Vancouver

Jeong H, Lee H, Kim C, Shin S. A Survey of Robot Intelligence with Large Language Models. Applied Sciences. 2024;14(19):8868. doi:10.3390/app14198868

BibTeX
@article{Jeong2024,
  author = {Jeong, Hyeongyo and Lee, Haechan and Kim, Changwon and Shin, Sungtae},
  title = {A Survey of Robot Intelligence with Large Language Models},
  journal = {Applied Sciences},
  year = {2024},
  volume = {14},
  number = {19},
  pages = {8868},
  doi = {10.3390/app14198868}
}
Kostenlos starten →