Large language models for scholarly ontology generation: An extensive analysis in the engineering field
Information Processing & Management · 2026-01 · 63 (1) · 104262
DOI: 10.1016/j.ipm.2025.104262 ↗Kurz gefasst
Fachartikel in der Zeitschrift Information Processing & Management, 2026-01. Verfasst von Tanay Aggarwal und 3 weiteren Autor:innen. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 98 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: COST.
Abstract (Originalsprache)
Ontologies of research topics are crucial for structuring scientific knowledge, enabling scientists to navigate vast amounts of research, and forming the backbone of intelligent systems such as search engines and recommendation systems. However, manual creation of these ontologies is expensive, slow, and often results in outdated and overly general representations. As a solution, researchers have been investigating ways to automate or semi-automate the process of generating these ontologies. This paper offers a comprehensive analysis of the ability of large language models (LLMs) to identify semantic relationships between different research topics, which is a critical step in the development of such ontologies. To this end, we developed a gold standard based on the IEEE Thesaurus to evaluate the task of identifying four types of relationships between pairs of topics: broader, narrower, same-as, and other. Our study evaluates the performance of seventeen LLMs, which differ in scale, accessibility (open vs. proprietary), and model type (full vs. quantised), while also assessing four zero-shot reasoning strategies. Several models have achieved outstanding results, including Mixtral-8x7B, Dolphin-Mistral-7B, and Claude 3 Sonnet, with F1-scores of 0.847, 0.920, and 0.967, respectively. Furthermore, our findings demonstrate that smaller, quantised models, when optimised through prompt engineering, can deliver performance comparable to much larger proprietary models, while requiring significantly fewer computational resources.
Förderung
- COST
Zitieren
APA
Aggarwal, T., Salatino, A., Osborne, F., & Motta, E. (2026). Large language models for scholarly ontology generation: An extensive analysis in the engineering field. Information Processing & Management, 63(1), 104262. https://doi.org/10.1016/j.ipm.2025.104262
Vancouver
Aggarwal T, Salatino A, Osborne F, Motta E. Large language models for scholarly ontology generation: An extensive analysis in the engineering field. Information Processing & Management. 2026;63(1):104262. doi:10.1016/j.ipm.2025.104262
BibTeX
@article{Aggarwal2026,
author = {Aggarwal, Tanay and Salatino, Angelo and Osborne, Francesco and Motta, Enrico},
title = {Large language models for scholarly ontology generation: An extensive analysis in the engineering field},
journal = {Information Processing & Management},
year = {2026},
volume = {63},
number = {1},
pages = {104262},
doi = {10.1016/j.ipm.2025.104262}
}
EUVIMED