EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Large language models for scholarly ontology generation: An extensive analysis in the engineering field

Tanay Aggarwal, Angelo Salatino, Francesco Osborne, Enrico Motta

Information Processing & Management · 2026-01 · 63 (1) · 104262

DOI: 10.1016/j.ipm.2025.104262 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information Processing & Management, 2026-01. Verfasst von Tanay Aggarwal und 3 weiteren Autor:innen. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 98 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: COST.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Ontologies of research topics are crucial for structuring scientific knowledge, enabling scientists to navigate vast amounts of research, and forming the backbone of intelligent systems such as search engines and recommendation systems. However, manual creation of these ontologies is expensive, slow, and often results in outdated and overly general representations. As a solution, researchers have been investigating ways to automate or semi-automate the process of generating these ontologies. This paper offers a comprehensive analysis of the ability of large language models (LLMs) to identify semantic relationships between different research topics, which is a critical step in the development of such ontologies. To this end, we developed a gold standard based on the IEEE Thesaurus to evaluate the task of identifying four types of relationships between pairs of topics: broader, narrower, same-as, and other. Our study evaluates the performance of seventeen LLMs, which differ in scale, accessibility (open vs. proprietary), and model type (full vs. quantised), while also assessing four zero-shot reasoning strategies. Several models have achieved outstanding results, including Mixtral-8x7B, Dolphin-Mistral-7B, and Claude 3 Sonnet, with F1-scores of 0.847, 0.920, and 0.967, respectively. Furthermore, our findings demonstrate that smaller, quantised models, when optimised through prompt engineering, can deliver performance comparable to much larger proprietary models, while requiring significantly fewer computational resources.

Förderung

  • COST

Zitieren

APA

Aggarwal, T., Salatino, A., Osborne, F., & Motta, E. (2026). Large language models for scholarly ontology generation: An extensive analysis in the engineering field. Information Processing & Management, 63(1), 104262. https://doi.org/10.1016/j.ipm.2025.104262

Vancouver

Aggarwal T, Salatino A, Osborne F, Motta E. Large language models for scholarly ontology generation: An extensive analysis in the engineering field. Information Processing & Management. 2026;63(1):104262. doi:10.1016/j.ipm.2025.104262

BibTeX
@article{Aggarwal2026,
  author = {Aggarwal, Tanay and Salatino, Angelo and Osborne, Francesco and Motta, Enrico},
  title = {Large language models for scholarly ontology generation: An extensive analysis in the engineering field},
  journal = {Information Processing & Management},
  year = {2026},
  volume = {63},
  number = {1},
  pages = {104262},
  doi = {10.1016/j.ipm.2025.104262}
}
Kostenlos starten →