Constructing a metadata knowledge graph as an atlas for demystifying AI pipeline optimization
Frontiers in Big Data · 07.01.2025 · 7
DOI: 10.3389/fdata.2024.1476506 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Big Data, erschienen am 07.01.2025. Verfasst von Revathy Venkataramanan und 10 weiteren Autor:innen. Beteiligte Einrichtung: AI Institute, University of South Carolina, Columbia, SC, United States. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The emergence of advanced artificial intelligence (AI) models has driven the development of frameworks and approaches that focus on automating model training and hyperparameter tuning of end-to-end AI pipelines. However, other crucial stages of these pipelines such as dataset selection, feature engineering, and model optimization for deployment have received less attention. Improving efficiency of end-to-end AI pipelines requires metadata of past executions of AI pipelines and all their stages. Regenerating metadata history by re-executing existing AI pipelines is computationally challenging and impractical. To address this issue, we propose to source AI pipeline metadata from open-source platforms such as Papers-with-Code, OpenML, and Hugging Face. However, integrating and unifying the varying terminologies and data formats from these diverse sources is a challenge. In this study, we present a solution by introducing Common Metadata Ontology (CMO) which is used to construct an extensive AI Pipeline Metadata Knowledge Graph (AIMKG) consisting of 1.6 million pipelines. Through semantic enhancements, the pipeline metadata in AIMKG is also enriched for downstream tasks such as search and recommendation of AI pipelines. We perform quantitative and qualitative evaluations on AIMKG to search and recommend relevant pipelines to user query. For quantitative evaluation, we propose a custom aggregation model that outperforms other baselines by achieving a retrieval accuracy (R@1) of 76.3%. Our qualitative analysis shows that AIMKG-based recommender retrieved relevant pipelines in 78% of test cases compared to the state-of-the-art MLSchema-based recommender which retrieved relevant responses in 51% of the cases. AIMKG serves as an atlas for navigating the evolving AI landscape, providing practitioners with a comprehensive factsheet for their applications. It guides AI pipeline optimization, offers insights and recommendations for improving AI pipelines, and serves as a foundation for data mining and analysis of evolving AI workflows.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- AI Institute, University of South Carolina, Columbia, SC, United States.
- Hewlett Packard Enterprise Labs, Houston, TX, United States.
Zitieren
APA
Venkataramanan, R., Tripathy, A., Kumar, T., Serebryakov, S., Justine, A., Shah, A., Bhattacharya, S., Foltin, M., Faraboschi, P., Roy, K., & Sheth, A. (2025). Constructing a metadata knowledge graph as an atlas for demystifying AI pipeline optimization. Frontiers in Big Data. https://doi.org/10.3389/fdata.2024.1476506
Vancouver
Venkataramanan R, Tripathy A, Kumar T, Serebryakov S, Justine A, Shah A, et al. Constructing a metadata knowledge graph as an atlas for demystifying AI pipeline optimization. Frontiers in Big Data. 2025. doi:10.3389/fdata.2024.1476506
BibTeX
@article{Venkataramanan2025,
author = {Venkataramanan, Revathy and Tripathy, Aalap and Kumar, Tarun and Serebryakov, Sergey and Justine, Annmary and Shah, Arpit and Bhattacharya, Suparna and Foltin, Martin and Faraboschi, Paolo and Roy, Kaushik and Sheth, Amit},
title = {Constructing a metadata knowledge graph as an atlas for demystifying AI pipeline optimization},
journal = {Frontiers in Big Data},
year = {2025},
doi = {10.3389/fdata.2024.1476506}
}
EUVIMED