EUVIMEDEUVIMED entdecken
Open Access

RAPID: Reliable and efficient Automatic generation of submission rePortIng checklists with large language moDels

Zeming Li, Xufei Luo, Zhenhua Yang, Huayu Zhang, Bingyi Wang, Long Ge iD, Zhaoxiang Bian, James Zou, Yaolong Chen iD, Lu Zhang iD, ADVANCED Working Group

Journal of the American Medical Informatics Association · 01.08.2025 · 32 (8) · 1340-1349

DOI: 10.1093/jamia/ocaf093 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 17.06.2025. Verfasst von Zeming Li und 10 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR 999077, China. Hauptthemen laut MEDLINE: Checklist, Artificial Intelligence, Research Report, Information Storage and Retrieval. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Young Collaborative Research, Health and Medical Research Fund, Guangdong-Hong Kong Technology.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To evaluate an automated reporting checklist generation tool using large language models and retrieval augmentation generation technology, called RAPID. Materials and Methods This study utilized large language models to develop a retrieval augmentation generation architecture. To assess its performance, a total of 91 published journal articles were collected and manually annotated in accordance with the CONSORT and CONSORT-AI medical reporting guidelines. These articles comprised 50 randomized controlled trials conducted without AI intervention and 41 randomized controlled trials that incorporated AI tools.

Results

Fifty RCT articles without the intervention of AI tools and 41 RCT articles with the intervention of AI tools were collected as CONSORT and CONSORT-AI datasets. All of the CONSORT reporting items (37) were included in the tool. RAPID achieved a high average accuracy rate of 92.11% and a content consistency score of 81.14% on the CONSORT dataset. Of the CONSORT-AI reporting items, 11 items related to the intervention of AI tools were included in the tool. RAPID achieved an average accuracy of 83.81% with a content consistency score of 72.51% on the CONSORT-AI dataset.

Discussion

RAPID may effectively save time and improve working efficiency for different user groups such as medical authors, researchers, editors, and reviewers.

Conclusion

RAPID has strong scalability, which can be easily adapted to different medical reporting guidelines without transfer learning on a large dataset. RAPID got state-of-the-art performance on 2 datasets for 2 different checklists compared to other methods.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ChecklistArtificial IntelligenceResearch ReportInformation Storage and RetrievalRandomized Controlled Trials as TopicHumansPeriodicals as TopicPublishingLarge Language Models

Stichwörter der Autor:innen

large language modelmedical reporting guidelinesrandomized clinical trialretrieval augmentation generation

Autor:innen und Institutionen

  • Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR 999077, China.
  • Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences (2021RU017), School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, China.
  • Vincent V.C. Woo Chinese Medicine Clinical Research Institute, School of Chinese Medicine, Hong Kong Baptist University, Hong Kong SAR 999077, China.
  • School of Public Health, Lanzhou University, Lanzhou 730000, China.
  • Chinese EQUATOR Centre, Hong Kong Baptist University, Hong Kong SAR 999077, China.
  • Department of Biomedical Data Science, Stanford University, Stanford, CA 94305, United States.

ORCID-Profile: Long Ge, Yaolong Chen, Lu Zhang

Förderung

  • Young Collaborative Research
  • Health and Medical Research Fund
  • Guangdong-Hong Kong Technology
  • Research Unit of Evidence-Based Evaluation and Guidelines
  • Chinese Academy of Medical Sciences
  • Vincent and Lily Woo Foundation
  • Guangdong-Hong Kong Technology Cooperation Funding Scheme
  • Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences

Zitieren

APA

Li, Z., Luo, X., Yang, Z., Zhang, H., Wang, B., Ge, L., Bian, Z., Zou, J., Chen, Y., Zhang, L., & ADVANCED Working Group (2025). RAPID: Reliable and efficient Automatic generation of submission rePortIng checklists with large language moDels. Journal of the American Medical Informatics Association, 32(8), 1340-1349. https://doi.org/10.1093/jamia/ocaf093

Vancouver

Li Z, Luo X, Yang Z, Zhang H, Wang B, Ge L, et al. RAPID: Reliable and efficient Automatic generation of submission rePortIng checklists with large language moDels. Journal of the American Medical Informatics Association. 2025;32(8):1340-1349. doi:10.1093/jamia/ocaf093

BibTeX
@article{Li2025,
  author = {Li, Zeming and Luo, Xufei and Yang, Zhenhua and Zhang, Huayu and Wang, Bingyi and Ge, Long and Bian, Zhaoxiang and Zou, James and Chen, Yaolong and Zhang, Lu and ADVANCED Working Group, ADVANCED Working Group},
  title = {RAPID: Reliable and efficient Automatic generation of submission rePortIng checklists with large language moDels},
  journal = {Journal of the American Medical Informatics Association},
  year = {2025},
  volume = {32},
  number = {8},
  pages = {1340--1349},
  doi = {10.1093/jamia/ocaf093}
}
Kostenlos starten →