Leveraging LLMs for Automated Extraction and Structuring of Educational Concepts and Relationships
Machine Learning and Knowledge Extraction · 19.09.2025 · 7 (3) · 103
DOI: 10.3390/make7030103 ↗Kurz gefasst
Fachartikel in der Zeitschrift Machine Learning and Knowledge Extraction, online erschienen am 19.09.2025. Verfasst von Tianyuan Yang und 5 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: JST SPRING, JSPS KAKENHI.
Abstract (Originalsprache)
Students must navigate large catalogs of courses and make appropriate enrollment decisions in many online learning environments. In this context, identifying key concepts and their relationships is essential for understanding course content and informing course recommendations. However, identifying and extracting concepts can be an extremely labor-intensive and time-consuming task when it has to be done manually. Traditional NLP-based methods to extract relevant concepts from courses heavily rely on resource-intensive preparation of detailed course materials, thereby failing to minimize labor. As recent advances in large language models (LLMs) offer a promising alternative for automating concept identification and relationship inference, we thoroughly investigate the potential of LLMs in automatically generating course concepts and their relations. Specifically, we systematically evaluate three LLM variants (GPT-3.5, GPT-4o-mini, and GPT-4o) across three distinct educational tasks, which are concept generation, concept extraction, and relation identification, using six systematically designed prompt configurations that range from minimal context (course title only) to rich context (course description, seed concepts, and subtitles). We systematically assess model performance through extensive automated experiments using standard metrics (Precision, Recall, F1, and Accuracy) and human evaluation by four domain experts, providing a comprehensive analysis of how prompt design and model choice influence the quality and reliability of the generated concepts and their interrelations. Our results show that GPT-3.5 achieves the highest scores on quantitative metrics, whereas GPT-4o and GPT-4o-mini often generate concepts that are more educationally meaningful despite lexical divergence from the ground truth. Nevertheless, LLM outputs still require expert revision, and performance is sensitive to prompt complexity. Overall, our experiments demonstrate the viability of LLMs as a tool for supporting educational content selection and delivery.
Förderung
- JST SPRING
- JSPS KAKENHI
Zitieren
APA
Yang, T., Ren, B., Gu, C., He, T., Ma, B., & Konomi, S. (2025). Leveraging LLMs for Automated Extraction and Structuring of Educational Concepts and Relationships. Machine Learning and Knowledge Extraction, 7(3), 103. https://doi.org/10.3390/make7030103
Vancouver
Yang T, Ren B, Gu C, He T, Ma B, Konomi S. Leveraging LLMs for Automated Extraction and Structuring of Educational Concepts and Relationships. Machine Learning and Knowledge Extraction. 2025;7(3):103. doi:10.3390/make7030103
BibTeX
@article{Yang2025,
author = {Yang, Tianyuan and Ren, Baofeng and Gu, Chenghao and He, Tianjia and Ma, Boxuan and Konomi, Shin’ichi},
title = {Leveraging LLMs for Automated Extraction and Structuring of Educational Concepts and Relationships},
journal = {Machine Learning and Knowledge Extraction},
year = {2025},
volume = {7},
number = {3},
pages = {103},
doi = {10.3390/make7030103}
}
EUVIMED