EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course

Stephen R Piccolo iD, Paul Denny, Andrew Luxton-Reilly iD, Samuel H Payne, Perry G Ridge

PLOS Computational Biology · 28.09.2023 · 19 (9) · e1011511

DOI: 10.1371/journal.pcbi.1011511 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Computational Biology, online erschienen am 28.09.2023. Verfasst von Stephen R Piccolo und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biology, Brigham Young University, Provo, Utah, United States of America. Hauptthemen laut MEDLINE: Computational Biology, Programming Languages. Die Arbeit wurde bisher 50-mal zitiert (Crossref) und nennt 74 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Computer programming is a fundamental tool for life scientists, allowing them to carry out essential research tasks. However, despite various educational efforts, learning to write code can be a challenging endeavor for students and researchers in life-sciences disciplines. Recent advances in artificial intelligence have made it possible to translate human-language prompts to functional code, raising questions about whether these technologies can aid (or replace) life scientists’ efforts to write code. Using 184 programming exercises from an introductory-bioinformatics course, we evaluated the extent to which one such tool—OpenAI’s ChatGPT—could successfully complete programming tasks. ChatGPT solved 139 (75.5%) of the exercises on its first attempt. For the remaining exercises, we provided natural-language feedback to the model, prompting it to try different approaches. Within 7 or fewer attempts, ChatGPT solved 179 (97.3%) of the exercises. These findings have implications for life-sciences education and research. Instructors may need to adapt their pedagogical approaches and assessment techniques to account for these new capabilities that are available to the general public. For some programming tasks, researchers may be able to work in collaboration with machine-learning models to produce functional code.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Computational BiologyProgramming LanguagesHumansArtificial IntelligenceSoftwareCurriculumLarge Language Models

Autor:innen und Institutionen

  • Department of Biology, Brigham Young University, Provo, Utah, United States of America.
  • School of Computer Science, The University of Auckland, Auckland, New Zealand.

ORCID-Profile: Stephen R Piccolo, Andrew Luxton-Reilly

Zitieren

APA

Piccolo, S. R., Denny, P., Luxton-Reilly, A., Payne, S. H., & Ridge, P. G. (2023). Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course. PLOS Computational Biology, 19(9), e1011511. https://doi.org/10.1371/journal.pcbi.1011511

Vancouver

Piccolo SR, Denny P, Luxton-Reilly A, Payne SH, Ridge PG. Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course. PLOS Computational Biology. 2023;19(9):e1011511. doi:10.1371/journal.pcbi.1011511

BibTeX
@article{Piccolo2023,
  author = {Piccolo, Stephen R and Denny, Paul and Luxton-Reilly, Andrew and Payne, Samuel H and Ridge, Perry G},
  title = {Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course},
  journal = {PLOS Computational Biology},
  year = {2023},
  volume = {19},
  number = {9},
  pages = {e1011511},
  doi = {10.1371/journal.pcbi.1011511}
}
Kostenlos starten →