EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

SEMA at LLMs4OL 2026 Task Taxonomy: Continual Pretraining on Serialized Ontologies for Cross-Domain Taxonomy Discovery

Miquel Canal Esteve, Yoan Gutiérrez, José Abreu Salas, Fernando Llopis

Open Conference Proceedings · 04.11.2026 · 11

DOI: 10.52825/ocp.v11i.3659 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Open Conference Proceedings, online erschienen am 04.11.2026. Verfasst von Miquel Canal Esteve und 3 weiteren Autor:innen. Die Arbeit nennt 15 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Agencia Estatal de Investigación, European Regional Development Fund, European Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper presents the SEMA submission to the Taxonomy Task of the third LLMs4OL Challenge. The task requires inducing is-a relations in unseen domains by transferring knowledge from ontologies in other domains. We investigate whether continual pretraining (CPT) on serialized ontological knowledge improves the cross-domain generalization of a compact open-weight language model. Starting from Llama 3.2–3B, we perform CPT on raw Turtle ontologies filtered to reduce contamination from held-out evaluation concepts. Checkpoints are retained after one and two CPT epochs and subsequently fine-tuned as binary parent–child classifiers. Supervised examples are built from source-domain taxonomy edges selected according to their semantic proximity to the target-domain vocabulary. Positive pairs are complemented with hard negatives that are semantically similar but not directly connected in the source taxonomies. We compare both CPT variants with an otherwise identical baseline initialized directly from Llama 3.2–3B, without continual pretraining. We evaluate our approach on two challenge domains, Hylex and Archeon. On Hylex, the best CPT configuration achieves an averaged F1 of 0.1602, compared with 0.1245 for the baseline. On Archeon, the best CPT model reaches 0.1493, compared with 0.1183 for the baseline. These results represent relative improvements of 28.7% and 26.2%, respectively. Overall, ontology-oriented CPT improves cross-domain taxonomy discovery, although the optimal number of CPT epochs differs across target domains.

Förderung

  • Agencia Estatal de Investigación
  • European Regional Development Fund
  • European Science Foundation

Zitieren

APA

Esteve, M. C., Gutiérrez, Y., Salas, J. A., & Llopis, F. (2026). SEMA at LLMs4OL 2026 Task Taxonomy: Continual Pretraining on Serialized Ontologies for Cross-Domain Taxonomy Discovery. Open Conference Proceedings, 11. https://doi.org/10.52825/ocp.v11i.3659

Vancouver

Esteve MC, Gutiérrez Y, Salas JA, Llopis F. SEMA at LLMs4OL 2026 Task Taxonomy: Continual Pretraining on Serialized Ontologies for Cross-Domain Taxonomy Discovery. Open Conference Proceedings. 2026;11. doi:10.52825/ocp.v11i.3659

BibTeX
@article{Esteve2026,
  author = {Esteve, Miquel Canal and Gutiérrez, Yoan and Salas, José Abreu and Llopis, Fernando},
  title = {SEMA at LLMs4OL 2026 Task Taxonomy: Continual Pretraining on Serialized Ontologies for Cross-Domain Taxonomy Discovery},
  journal = {Open Conference Proceedings},
  year = {2026},
  volume = {11},
  doi = {10.52825/ocp.v11i.3659}
}
Kostenlos starten →