EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Leveraging artificial intelligence to summarize abstracts in lay language for increasing research accessibility and transparency

Cathy Shyr iD, Randall W Grout iD, Nan Kennedy, Yasemin Akdas, Maeve Tischbein, Joshua Milford, Jason Tan, Kaysi Quarles, Terri L Edwards, Laurie L Novak, Jules White, Consuelo H Wilkins und 1 weitere

Journal of the American Medical Informatics Association · 2024

DOI: 10.1093/jamia/ocae186 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, 2024. Verfasst von Cathy Shyr und 12 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States. Hauptthemen laut MEDLINE: Artificial Intelligence, Abstracting and Indexing. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: NLM NIH HHS, NCATS NIH HHS, National Center for Advancing Translational Sciences.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

Returning aggregate study results is an important ethical responsibility to promote trust and inform decision making, but the practice of providing results to a lay audience is not widely adopted. Barriers include significant cost and time required to develop lay summaries and scarce infrastructure necessary for returning them to the public. Our study aims to generate, evaluate, and implement ChatGPT 4 lay summaries of scientific abstracts on a national clinical study recruitment platform, ResearchMatch, to facilitate timely and cost-effective return of study results at scale. Materials and Methods We engineered prompts to summarize abstracts at a literacy level accessible to the public, prioritizing succinctness, clarity, and practical relevance. Researchers and volunteers assessed ChatGPT-generated lay summaries across five dimensions: accuracy, relevance, accessibility, transparency, and harmfulness. We used precision analysis and adaptive random sampling to determine the optimal number of summaries for evaluation, ensuring high statistical precision.

Results

ChatGPT achieved 95.9% (95% CI, 92.1–97.9) accuracy and 96.2% (92.4–98.1) relevance across 192 summary sentences from 33 abstracts based on researcher review. 85.3% (69.9–93.6) of 34 volunteers perceived ChatGPT-generated summaries as more accessible and 73.5% (56.9–85.4) more transparent than the original abstract. None of the summaries were deemed harmful. We expanded ResearchMatch’s technical infrastructure to automatically generate and display lay summaries for over 750 published studies that resulted from the platform’s recruitment mechanism. Discussion and Conclusion Implementing AI-generated lay summaries on ResearchMatch demonstrates the potential of a scalable framework generalizable to broader platforms for enhancing research accessibility and transparency.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceAbstracting and IndexingHumansBiomedical ResearchInformation Dissemination

Stichwörter der Autor:innen

ResearchMatchartificial intelligencelarge language modelreturn of study resultstext summarization

Autor:innen und Institutionen

  • Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
  • Department of Pediatrics, Indiana University School of Medicine, Indianapolis, IN 46202, United States.
  • Regenstrief Institute, Inc, Indianapolis, IN 46202, United States.
  • Vanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, TN 37203, United States.
  • Division of Emergency Medicine Research, Vanderbilt University Medical Center, Nashville, TN 37232, United States.
  • Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN 37240, United States.

ORCID-Profile: Cathy Shyr, Randall W Grout

Förderung

  • NLM NIH HHS
  • NCATS NIH HHS
  • National Center for Advancing Translational Sciences
  • National Library of Medicine

Zitieren

APA

Shyr, C., Grout, R. W., Kennedy, N., Akdas, Y., Tischbein, M., Milford, J., Tan, J., Quarles, K., Edwards, T. L., Novak, L. L., White, J., Wilkins, C. H., & Harris, P. A. (2024). Leveraging artificial intelligence to summarize abstracts in lay language for increasing research accessibility and transparency. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocae186

Vancouver

Shyr C, Grout RW, Kennedy N, Akdas Y, Tischbein M, Milford J, et al. Leveraging artificial intelligence to summarize abstracts in lay language for increasing research accessibility and transparency. Journal of the American Medical Informatics Association. 2024. doi:10.1093/jamia/ocae186

BibTeX
@article{Shyr2024,
  author = {Shyr, Cathy and Grout, Randall W and Kennedy, Nan and Akdas, Yasemin and Tischbein, Maeve and Milford, Joshua and Tan, Jason and Quarles, Kaysi and Edwards, Terri L and Novak, Laurie L and White, Jules and Wilkins, Consuelo H and Harris, Paul A},
  title = {Leveraging artificial intelligence to summarize abstracts in lay language for increasing research accessibility and transparency},
  journal = {Journal of the American Medical Informatics Association},
  year = {2024},
  doi = {10.1093/jamia/ocae186}
}
Kostenlos starten →