Agentic Data Pipelines: Autonomous ELT Orchestration Using AI Agents on Microsoft Fabric and Databricks
International Journal of Computer Technology and Electronics Communication ( · 24.12.2025 · 08 (06)
DOI: 10.15680/ijctece.2025.0806034 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Technology and Electronics Communication (, online erschienen am 24.12.2025. Verfasst von Narendra Mangala.
Abstract (Originalsprache)
ELT workflows orchestrated by modern cloud services can be automated using agent-based frameworks that create and execute the required solutions. Agents working in a multi-agent fashion allow for tools such as Microsoft Fabric and Azure Data Factory to be used in novel ways. An evaluation of such an architecture, focused primarily on the agent-based approach within the Microsoft ecosystem, demonstrated that a complete pipeline could be processed as a single task on Databricks. Each agent’s specialization influenced not just the chosen tools but also the orchestration pattern. The decisions taken at these levels were logged as text and compared with human counterparts, providing insight about the explainability of the different patterns. Autonomous orchestration of ELT solutions using Microsoft Fabric Data Factory has proven successful and is paving the way toward an unassisted data engineering process. Support for these workers through user-defined tasks might be useful for more complex cases.
Zitieren
APA
Mangala, N. (2025). Agentic Data Pipelines: Autonomous ELT Orchestration Using AI Agents on Microsoft Fabric and Databricks. International Journal of Computer Technology and Electronics Communication (, 08(06). https://doi.org/10.15680/ijctece.2025.0806034
Vancouver
Mangala N. Agentic Data Pipelines: Autonomous ELT Orchestration Using AI Agents on Microsoft Fabric and Databricks. International Journal of Computer Technology and Electronics Communication (. 2025;08(06). doi:10.15680/ijctece.2025.0806034
BibTeX
@article{Mangala2025,
author = {Mangala, Narendra},
title = {Agentic Data Pipelines: Autonomous ELT Orchestration Using AI Agents on Microsoft Fabric and Databricks},
journal = {International Journal of Computer Technology and Electronics Communication (},
year = {2025},
volume = {08},
number = {06},
doi = {10.15680/ijctece.2025.0806034}
}
EUVIMED