Go to AIMS 2025 Workshop T2P homepageA Multi-Criteria Automated MLOps Pipeline for Cost-Effective Cloud-Based Classifier Retraining in Response to Data Distribution Shifts
Keywords: MLOps, Data distribution Drift, Classification tasks.
Abstract: The performance of machine learning (ML) models often deteriorates when the underlying data distribution changes over time, a
phenomenon known as data distribution drift. When this happens, ML models need to be retrained and redeployed. This MLOps is often manual, i.e., humans trigger the process of model retraining and redeployment. In this work, we present an automated MLOps pipeline designed to address neural network classifier retraining in response to significant data distribution changes. Our MLOps pipeline employs multi-criteria statistical techniques to detect distribution shifts and triggers model updates only when necessary, ensuring computational efficiency and resource optimization. We demonstrate the effectiveness of our framework through experiments on several benchmark anomaly detection datasets, showing significant improvements in model accuracy and robustness compared to traditional retraining strategies. Our work provides a foundation for deploying more reliable and adaptive machine learning systems in dynamic real-world settings, where data distribution changes are common.
Submission Number: 1
Loading