Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course
PLOS Computational Biology · 28.09.2023 · 19 (9) · e1011511
DOI: 10.1371/journal.pcbi.1011511 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Computational Biology, online erschienen am 28.09.2023. Verfasst von Stephen R Piccolo und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biology, Brigham Young University, Provo, Utah, United States of America. Hauptthemen laut MEDLINE: Computational Biology, Programming Languages. Die Arbeit wurde bisher 50-mal zitiert (Crossref) und nennt 74 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Computer programming is a fundamental tool for life scientists, allowing them to carry out essential research tasks. However, despite various educational efforts, learning to write code can be a challenging endeavor for students and researchers in life-sciences disciplines. Recent advances in artificial intelligence have made it possible to translate human-language prompts to functional code, raising questions about whether these technologies can aid (or replace) life scientists’ efforts to write code. Using 184 programming exercises from an introductory-bioinformatics course, we evaluated the extent to which one such tool—OpenAI’s ChatGPT—could successfully complete programming tasks. ChatGPT solved 139 (75.5%) of the exercises on its first attempt. For the remaining exercises, we provided natural-language feedback to the model, prompting it to try different approaches. Within 7 or fewer attempts, ChatGPT solved 179 (97.3%) of the exercises. These findings have implications for life-sciences education and research. Instructors may need to adapt their pedagogical approaches and assessment techniques to account for these new capabilities that are available to the general public. For some programming tasks, researchers may be able to work in collaboration with machine-learning models to produce functional code.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Biology, Brigham Young University, Provo, Utah, United States of America.
- School of Computer Science, The University of Auckland, Auckland, New Zealand.
ORCID-Profile: Stephen R Piccolo, Andrew Luxton-Reilly
Zitieren
APA
Piccolo, S. R., Denny, P., Luxton-Reilly, A., Payne, S. H., & Ridge, P. G. (2023). Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course. PLOS Computational Biology, 19(9), e1011511. https://doi.org/10.1371/journal.pcbi.1011511
Vancouver
Piccolo SR, Denny P, Luxton-Reilly A, Payne SH, Ridge PG. Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course. PLOS Computational Biology. 2023;19(9):e1011511. doi:10.1371/journal.pcbi.1011511
BibTeX
@article{Piccolo2023,
author = {Piccolo, Stephen R and Denny, Paul and Luxton-Reilly, Andrew and Payne, Samuel H and Ridge, Perry G},
title = {Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course},
journal = {PLOS Computational Biology},
year = {2023},
volume = {19},
number = {9},
pages = {e1011511},
doi = {10.1371/journal.pcbi.1011511}
}
EUVIMED