EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

LLM-Assisted Annotation for Spanish Medical Corpus Construction: An Empirical Evaluation

Ovidia Martínez Sánchez, Borja Navarro Colorado

Journal of Computer-Assisted Linguistic Research · 18.11.2026 · 10 · 1-28

DOI: 10.4995/jclr.2026.26224 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Computer-Assisted Linguistic Research, online erschienen am 18.11.2026. Verfasst von Ovidia Martínez Sánchez und 1 weiteren Autor:innen. Die Arbeit nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Generalitat Valenciana, Agencia Estatal de Investigación.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Named entity annotation in clinical text is a costly and time-intensive task, particularly for low-resource languages such as Spanish. This article introduces CLINFEM, an XML-annotated corpus comprising 1,200 Spanish medical reports from two regions of Spain. CLINFEM was created using a thorough human-in-the-loop annotation process combining large language model pre-annotation and systematic expert review. It is imperative to acknowledge that each of the 1,200 medical reports underwent meticulous review and correction by a trained human expert. Large language model pre-annotation was employed solely for the purpose of saving time and was never intended to serve as a substitute for manual annotation. Consequently, the entire CLINFEM corpus is a human-validated, gold-standard resource. GPT-4o and Gemini 1.5 Pro were evaluated as pre-annotators using a custom scheme comprising eight clinical entity types and one relational category. GPT-4o achieved a Cohen's κ of 0.73 against human annotation, while Gemini 1.5 Pro obtained κ = 0.41. Human expert review yielded an overall inter-annotator agreement of κ = 0.869 (F1 = 0.793), with a gold-standard subset agreement of κ = 0.9894. This approach reduced the overall annotation effort by around 74% compared to a fully manual approach (81% for the bulk phase alone), saving over 1,000 hours. Error rates varied by medical speciality, with oncology requiring the highest correction rate (94.3% of reports). These results demonstrate that using large language models to assist annotation is a viable and efficient strategy for constructing clinical corpora in Spanish, while also highlighting the continued necessity of human oversight.

Förderung

  • Generalitat Valenciana
  • Agencia Estatal de Investigación

Zitieren

APA

Sánchez, O. M., & Colorado, B. N. (2026). LLM-Assisted Annotation for Spanish Medical Corpus Construction: An Empirical Evaluation. Journal of Computer-Assisted Linguistic Research, 10, 1-28. https://doi.org/10.4995/jclr.2026.26224

Vancouver

Sánchez OM, Colorado BN. LLM-Assisted Annotation for Spanish Medical Corpus Construction: An Empirical Evaluation. Journal of Computer-Assisted Linguistic Research. 2026;10:1-28. doi:10.4995/jclr.2026.26224

BibTeX
@article{Snchez2026,
  author = {Sánchez, Ovidia Martínez and Colorado, Borja Navarro},
  title = {LLM-Assisted Annotation for Spanish Medical Corpus Construction: An Empirical Evaluation},
  journal = {Journal of Computer-Assisted Linguistic Research},
  year = {2026},
  volume = {10},
  pages = {1--28},
  doi = {10.4995/jclr.2026.26224}
}
Kostenlos starten →