Automated Data Quality Gates for AI Training Pipelines
American Journal of Technology · 2025
DOI: 10.58425/ajt.v4i3.455 ↗Kurz gefasst
Fachartikel in der Zeitschrift American Journal of Technology, 2025. Verfasst von Vijayalakshmi Singavarapu. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Aim
The aim of this study is to design and evaluate automated data quality gates integrated within AI training pipelines in order to improve data validation efficiency and model accuracy. A further objective is to automate the data validation process, which has traditionally relied on manual tests or slow, error-prone rule-checking applications that are difficult to extend.
Methods
The proposed automated data quality gate system was implemented and tested using multiple open-source datasets. Its performance was evaluated by assessing model accuracy, error rates, and the time required for data validation within AI training pipelines.
Results
The findings show that automated data quality gates reduce the likelihood of errors introduced during the training process and improve model accuracy by 10 - 30%. The system also reduces model error rates by 10 - 20% and significantly accelerates data validation, decreasing validation time from approximately 5 hours to less than 1 hour. Overall, the system demonstrated the ability to improve model performance by 15 - 25% compared to regular training methods while ensuring that only high-quality data flows into AI pipelines. These improvements enhance the scalability and accuracy of AI systems across application domains.
Conclusion
The study concludes that automated data quality gates are effective in improving data validation efficiency and model accuracy. By automating data validation, the proposed approach overcomes limitations associated with manual and rule-based validation methods, leading to more reliable and scalable AI training pipelines suitable for deployment in various industries. Recommendations: The study recommends the wider adoption of automated data quality gates across AI applications, particularly in critical sectors such as healthcare and finance.
Zitieren
APA
Singavarapu, V. (2025). Automated Data Quality Gates for AI Training Pipelines. American Journal of Technology. https://doi.org/10.58425/ajt.v4i3.455
Vancouver
Singavarapu V. Automated Data Quality Gates for AI Training Pipelines. American Journal of Technology. 2025. doi:10.58425/ajt.v4i3.455
BibTeX
@article{Singavarapu2025,
author = {Singavarapu, Vijayalakshmi},
title = {Automated Data Quality Gates for AI Training Pipelines},
journal = {American Journal of Technology},
year = {2025},
doi = {10.58425/ajt.v4i3.455}
}
EUVIMED