EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 3.0

Accuracy, Reliability, and Bloom’s Taxonomy Performance of Seven Large Language Models on Microbiology Questions

Volodymyr Dvornyk iD, Olena Bolgova iD, Volodymyr Mavrych iD

Advances in Medical Education and Practice · 2026-08 · Volume 17 · 1-11

DOI: 10.2147/amep.s621664 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Advances in Medical Education and Practice, 2026-08. Verfasst von Volodymyr Dvornyk und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Life Sciences, College of Science and General Studies, Alfaisal University, Riyadh, 11533, Saudi Arabia. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 3.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) are increasingly used as learning resources in medical education, yet their performance and reliability in microbiology, a discipline with a broad, heterogeneous knowledge base, have not been systematically evaluated across multiple platforms.

Objective

To benchmark seven publicly available LLMs on microbiology multiple-choice questions (MCQs), assessing overall accuracy, test-retest reliability, topic-specific performance, and the relationship between cognitive complexity and model performance.

Methods

Seven LLMs (Claude 4.6 Sonnet, Gemini 3.0, ChatGPT-5.2, Grok 4, Copilot, DeepSeek V3, and Kimi K2) completed 200 MCQs distributed across 20 microbiology topics and five Bloom's taxonomy levels in three independent sessions separated by 24-hour intervals. A total of 4200 responses were analyzed. Statistical analysis included one-way ANOVA with Tukey's HSD post hoc tests, repeated-measures ANOVA, intraclass correlation coefficients (ICCs), and Pearson correlations.

Results

The collective mean accuracy was 86.18%. Six of seven systems exceeded the 80% high-competency threshold; Claude (89.83%), Grok (89.50%), and GPT (88.83%) led the group. Gemini (72.33%) was the only underperforming system. Test-retest reliability varied dramatically: Claude achieved excellent ICC (0.966), while Gemini exhibited poor reliability (ICC = 0.290), with session-to-session fluctuations of up to 100 percentage points on individual topics. Microbial Cell (100%) was the easiest topic; Viral Genomics (61.9%) was the most challenging across all systems. A uniform decline at Bloom's Level 4 (Analyze) was observed across all LLMs, with no model exceeding 78%.

Conclusion

Contemporary LLMs demonstrate substantial knowledge of microbiology but differ markedly in reliability. Response consistency, alongside accuracy, should be a primary criterion for educational deployment. These findings are specific to microbiology MCQ performance and may not generalize to open-ended clinical reasoning.

Schlagwörter

Stichwörter der Autor:innen

artificial intelligencebloom’s taxonomylarge language modelsmicrobiology educationtest-retest reliability

Autor:innen und Institutionen

  • Department of Life Sciences, College of Science and General Studies, Alfaisal University, Riyadh, 11533, Saudi Arabia.
  • Department of Anatomy and Genetics, College of Medicine, Alfaisal University, Riyadh, 11533, Saudi Arabia.

ORCID-Profile: Volodymyr Dvornyk, Olena Bolgova, Volodymyr Mavrych

Zitieren

APA

Dvornyk, V., Bolgova, O., & Mavrych, V. (2026). Accuracy, Reliability, and Bloom’s Taxonomy Performance of Seven Large Language Models on Microbiology Questions. Advances in Medical Education and Practice, Volume 17, 1-11. https://doi.org/10.2147/amep.s621664

Vancouver

Dvornyk V, Bolgova O, Mavrych V. Accuracy, Reliability, and Bloom’s Taxonomy Performance of Seven Large Language Models on Microbiology Questions. Advances in Medical Education and Practice. 2026;Volume 17:1-11. doi:10.2147/amep.s621664

BibTeX
@article{Dvornyk2026,
  author = {Dvornyk, Volodymyr and Bolgova, Olena and Mavrych, Volodymyr},
  title = {Accuracy, Reliability, and Bloom’s Taxonomy Performance of Seven Large Language Models on Microbiology Questions},
  journal = {Advances in Medical Education and Practice},
  year = {2026},
  volume = {Volume 17},
  pages = {1--11},
  doi = {10.2147/amep.s621664}
}
Kostenlos starten →