Transforming role classification in scientific teams using LLMs and advanced predictive analytics
Quantitative Science Studies · 12.05.2025 · 6 · 505-523
DOI: 10.1162/qss_a_00360 ↗Kurz gefasst
Fachartikel in der Zeitschrift Quantitative Science Studies, online erschienen am 12.05.2025. Verfasst von Wonduk Seo und 1 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, 2024 Cultural Research Project of Ningbo.
Abstract (Originalsprache)
Scientific team dynamics are critical in determining the nature and impact of research outputs. However, existing methods for classifying author roles based on self-reports and clustering lack comprehensive contextual analysis of contributions. Thus, we present a transformative approach to classifying author roles in scientific teams using advanced large language models (LLMs), which offers a more refined analysis compared to traditional clustering methods. Specifically, we seek to complement and enhance these traditional methods by utilizing open source and proprietary LLMs, such as GPT-4, Llama3 70B, Llama2 70B, and Mistral 7x8B, for role classification. Utilizing few-shot prompting, we categorize author roles and demonstrate that GPT-4 outperforms other models across multiple categories, surpassing traditional approaches such as XGBoost and BERT. Our methodology also includes building a predictive deep learning model using 10 features. By training this model on a data set derived from the OpenAlex database, which provides detailed metadata on academic publications—such as author-publication history, author affiliation, research topics, and citation counts—we achieve an F1 score of 0.76, demonstrating robust classification of author roles.
Förderung
- National Natural Science Foundation of China
- 2024 Cultural Research Project of Ningbo
Zitieren
APA
Seo, W., & Bu, Y. (2025). Transforming role classification in scientific teams using LLMs and advanced predictive analytics. Quantitative Science Studies, 6, 505-523. https://doi.org/10.1162/qss_a_00360
Vancouver
Seo W, Bu Y. Transforming role classification in scientific teams using LLMs and advanced predictive analytics. Quantitative Science Studies. 2025;6:505-523. doi:10.1162/qss_a_00360
BibTeX
@article{Seo2025,
author = {Seo, Wonduk and Bu, Yi},
title = {Transforming role classification in scientific teams using LLMs and advanced predictive analytics},
journal = {Quantitative Science Studies},
year = {2025},
volume = {6},
pages = {505--523},
doi = {10.1162/qss_a_00360}
}
EUVIMED