EUVIMEDEUVIMED entdecken
Evaluationsstudie

The Role of Artificial Intelligence Large Language Models in Literature Search Assistance to Evaluate Inguinal Hernia Repair Approaches

Joao P G Kasakewitch, Diego L Lima iD, Carlos A Balthazar da Silveira iD, Valberto Sanha, Ana Caroline Rasador, Leandro Totti Cavazzola, Julio Mayol, Flavio Malcher

Journal of Laparoendoscopic & Advanced Surgical Techniques · 01.06.2025 · 35 (6) · 437-444

DOI: 10.1089/lap.2024.0277 ↗

Kurz gefasst

Evaluationsstudie in der Zeitschrift Journal of Laparoendoscopic & Advanced Surgical Techniques, online erschienen am 10.06.2025. Verfasst von Joao P G Kasakewitch und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Surgery, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA. Hauptthemen laut MEDLINE: Artificial Intelligence, Hernia, Inguinal, Herniorrhaphy, Information Storage and Retrieval. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 21 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aim

This study assesses the reliability of artificial intelligence (AI) large language models (LLMs) in identifying relevant literature comparing inguinal hernia repair techniques. Material and Methods: We used LLM chatbots (Bing Chat AI, ChatGPT versions 3.5 and 4.0, and Gemini) to find comparative studies and randomized controlled trials on inguinal hernia repair techniques. The results were then compared with existing systematic reviews (SRs) and meta-analyses and checked for the authenticity of listed articles.

Results

LLMs screened 22 studies from 2006 to 2023 across eight journals, while the SRs encompassed a total of 42 studies. Through thorough external validation, 63.6% of the studies (14 out of 22), including 10 identified through Chat GPT 4.0 and 6 via Bing AI (with an overlap of 2 studies between them), were confirmed to be authentic. Conversely, 36.3% (8 out of 22) were revealed as fabrications by Google Gemini (Bard), with two (25.0%) of these fabrications mistakenly linked to valid DOIs. Four (25.6%) of the 14 real studies were acknowledged in the SRs, which represents 18.1% of all LLM-generated studies. LLMs missed a total of 38 (90.5%) of the studies included in the previous SRs, while 10 real studies were found by the LLMs but were not included in the previous SRs. Between those 10 studies, 6 were reviews, and 1 was published after the SRs, leaving a total of three comparative studies missed by the reviews.

Conclusions

This study reveals the mixed reliability of AI language models in scientific searches. Emphasizing a cautious application of AI in academia and the importance of continuous evaluation of AI tools in scientific investigations.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceHernia, InguinalHerniorrhaphyInformation Storage and RetrievalHumansLarge Language ModelsRandomized Controlled Trials as TopicReproducibility of ResultsSystematic Reviews as TopicMeta-Analysis as Topic

Stichwörter der Autor:innen

artificial intelligenceinguinal hernialaparoscopic surgeryminimally invasive surgeryopen surgeryrobotic surgery

Autor:innen und Institutionen

  • Department of Surgery, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA.
  • Department of Surgery, Montefiore Medical Center, The Bronx, New York, USA.
  • Escola Bahiana de Medicina e Saúde Pública, Salvador, Brasil.
  • Department of Surgery, Universidade Federal de Ciências da Saúde de Porto Alegre, Porto Alegre, Brasil.
  • Department of Surgery, Universidade Federal do Rio Grande Do Sul, Porto Alegre, Brasil.
  • Hospital Clínico San Carlos, IdISSC, Universidad Complutense de Madrid, Madrid, Spain.

ORCID-Profile: Diego L Lima, Carlos A Balthazar da Silveira

Zitieren

APA

Kasakewitch, J. P. G., Lima, D. L., Balthazar da Silveira, C. A., Sanha, V., Rasador, A. C., Cavazzola, L. T., Mayol, J., & Malcher, F. (2025). The Role of Artificial Intelligence Large Language Models in Literature Search Assistance to Evaluate Inguinal Hernia Repair Approaches. Journal of Laparoendoscopic & Advanced Surgical Techniques, 35(6), 437-444. https://doi.org/10.1089/lap.2024.0277

Vancouver

Kasakewitch JPG, Lima DL, Balthazar da Silveira CA, Sanha V, Rasador AC, Cavazzola LT, et al. The Role of Artificial Intelligence Large Language Models in Literature Search Assistance to Evaluate Inguinal Hernia Repair Approaches. Journal of Laparoendoscopic & Advanced Surgical Techniques. 2025;35(6):437-444. doi:10.1089/lap.2024.0277

BibTeX
@article{Kasakewitch2025,
  author = {Kasakewitch, Joao P G and Lima, Diego L and Balthazar da Silveira, Carlos A and Sanha, Valberto and Rasador, Ana Caroline and Cavazzola, Leandro Totti and Mayol, Julio and Malcher, Flavio},
  title = {The Role of Artificial Intelligence Large Language Models in Literature Search Assistance to Evaluate Inguinal Hernia Repair Approaches},
  journal = {Journal of Laparoendoscopic & Advanced Surgical Techniques},
  year = {2025},
  volume = {35},
  number = {6},
  pages = {437--444},
  doi = {10.1089/lap.2024.0277}
}
Kostenlos starten →