Optimizing Clinical Temporal Relation Extraction With Large Language Models: Comparative Analysis
JMIR Medical Informatics · 15.10.2026 · 14 · e89957-e89957
DOI: 10.2196/89957 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift JMIR Medical Informatics, online erschienen am 15.10.2026. Verfasst von Jianping He und 7 weiteren Autor:innen. Beteiligte Einrichtung: McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, United States. Hauptthemen laut MEDLINE: Large Language Models, Natural Language Processing, Electronic Health Records. Die Arbeit nennt 58 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIA NIH HHS, NCCIH NIH HHS, NLM NIH HHS.
Abstract (Originalsprache)
Background
Clinical temporal relation extraction (CTRE) is essential for reconstructing patient timelines from unstructured electronic health records (EHRs). However, the linguistic complexity of clinical notes and the high cost of expert annotation impede the development of large-scale training corpora. While large language models (LLMs) have transformed general natural language processing, their application to CTRE remains underexplored.
Objective
This study aimed to determine the optimal adaptation strategy for CTRE by conducting a controlled comparative benchmarking of LLM architectures and fine-tuning methodologies in both data-rich and limited-data regimes.
Methods
We evaluated 4 LLMs representing 2 distinct architectures—transformer encoders (GatorTron [University of Florida and NVIDIA]) and transformer decoders (LLaMA 3.1-8B [Meta AI] and MeLLaMA-13B [Yale University and University of Florida]). We compared four adaptation strategies: (1) standard fine-tuning, (2) hard-prompting, (3) soft-prompting, and (4) low-rank adaptation (LoRA). Experiments were conducted on the 2012 i2b2 CTRE benchmark in both full-supervision and 1-shot scenarios.
Results
Recent LLMs combined with appropriate adaptation strategies achieved strong performance for CTRE on the 2012 i2b2 benchmark. In the data-rich setting, the best-performing configuration was LLaMA3.1-8B with quantized low-rank adaptation (QLoRA) applied to the query and value projection layers, achieving a micro- F 1 -score of 89.15% (0.30). Comparative analyses showed that QLoRA applied only to the query and value projection layers achieved performance comparable to QLoRA applied to all linear layers, while requiring fewer trainable parameters. Hard-prompting showed particular effectiveness in the 1-shot scenario compared with standard fine-tuning for GatorTron-Base. Overall, Decoder-based models generally outperformed encoder-based models under QLoRA-based adaptation. However, under the most resource-constrained setting, where both training data and trainable parameters were highly limited, encoder-based models achieved better performance than decoder-based models.
Conclusions
This study provides a controlled benchmark for adapting LLMs to CTRE under different data and parameter constraints. Our findings suggest that QLoRA targeting only the query and value projection layers is a computationally efficient and competitive adaptation strategy. Decoder-based LLMs generally achieved stronger performance under QLoRA-based adaptation, whereas domain-specific encoder-based models showed advantages under the most resource-constrained soft-prompting setting. These results highlight the importance of selecting adaptation strategies according to both model architecture and available computational resources.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, United States.
- Department of Artificial Intelligence and Informatics, Mayo Clinic in Florida, 4500 San Pablo Rd S, Jacksonville, FL, 32224, United States, 1 904 956 3256.
Förderung
- NIA NIH HHS
- NCCIH NIH HHS
- NLM NIH HHS
Zitieren
APA
He, J., Sun, Z., Rasmy, L., Li, H., Li, J., Yu, E., Zhi, D., & Tao, C. (2026). Optimizing Clinical Temporal Relation Extraction With Large Language Models: Comparative Analysis. JMIR Medical Informatics, 14, e89957-e89957. https://doi.org/10.2196/89957
Vancouver
He J, Sun Z, Rasmy L, Li H, Li J, Yu E, et al. Optimizing Clinical Temporal Relation Extraction With Large Language Models: Comparative Analysis. JMIR Medical Informatics. 2026;14:e89957-e89957. doi:10.2196/89957
BibTeX
@article{He2026,
author = {He, Jianping and Sun, Zenan and Rasmy, Laila and Li, Haifang and Li, Jianfu and Yu, Evan and Zhi, Degui and Tao, Cui},
title = {Optimizing Clinical Temporal Relation Extraction With Large Language Models: Comparative Analysis},
journal = {JMIR Medical Informatics},
year = {2026},
volume = {14},
pages = {e89957--e89957},
doi = {10.2196/89957}
}
EUVIMED