EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating large language models in biomedical data science challenges through a classroom experiment

Huifang Ma, BIOSTAT 824 Student Consortium, Zhicheng Ji iD

Proceedings of the National Academy of Sciences · 2025

DOI: 10.1073/pnas.2521062122 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the National Academy of Sciences, 2025. Verfasst von Huifang Ma und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC 27705. Hauptthemen laut MEDLINE: Data Science, Language. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) have shown remarkable capabilities in algorithm design, but their effectiveness in solving data science challenges in real-world settings remains poorly understood. We conducted a classroom experiment in which graduate students used LLMs to solve biomedical data science challenges on Kaggle, focusing on tabular data prediction. While their submissions did not top the leaderboards, their prediction scores were often close to those of leading human participants. LLMs frequently recommended gradient boosting methods, which were associated with better performance. Among prompting strategies, self-refinement, where the LLM improves its own initial solution, was the most effective, a result validated using additional LLMs. While LLMs are capable of handling more complex data science tasks beyond tabular data prediction, their performance is substantially worse. These findings demonstrate that LLMs have the potential to design competitive machine learning solutions, even when used by nonexperts.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Data ScienceLanguageHumansMachine LearningAlgorithmsLarge Language Models

Stichwörter der Autor:innen

data sciencelarge language modelmachine learning

Autor:innen und Institutionen

  • Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC 27705.

ORCID-Profile: Zhicheng Ji

Zitieren

APA

Ma, H., BIOSTAT 824 Student Consortium, & Ji, Z. (2025). Evaluating large language models in biomedical data science challenges through a classroom experiment. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2521062122

Vancouver

Ma H, BIOSTAT 824 Student Consortium, Ji Z. Evaluating large language models in biomedical data science challenges through a classroom experiment. Proceedings of the National Academy of Sciences. 2025. doi:10.1073/pnas.2521062122

BibTeX
@article{Ma2025,
  author = {Ma, Huifang and BIOSTAT 824 Student Consortium, BIOSTAT 824 Student Consortium and Ji, Zhicheng},
  title = {Evaluating large language models in biomedical data science challenges through a classroom experiment},
  journal = {Proceedings of the National Academy of Sciences},
  year = {2025},
  doi = {10.1073/pnas.2521062122}
}
Kostenlos starten →