MLOps Maturity Model

MLOps is a set of tools and practices which help to deploy and maintain machine learning models in a production environment.

The MLOps Maturity Model aims at clarifying DevOps principles and practices necessary to run a successful necessary MLOps environment

Level 0: No MLOps

At this level, things are done manually. We manually gather the data. The preprocessing, model training and other compute-related activities are also likely manuals and not managed. There is no experiment tracking. The end result would most likely be a model file which will be used. The difference would most likely be tracked in an Excel worksheet or text file manually. The testing would also likely be manual.

This type of process is most useful for POCs and Data Scientists where the usage is very limited

Level 1: DevOps no MLOps

Here the data will be gathered automatically through the data pipeline. Tools like Azure Data Factory, and SnowFlake could be used here. The processes regarding training, testing and experiment tracking are similar to level 0. Here the model will be given to the software engineers who will be using CICD pipelines to deploy the model

Level 2: Automated Training

Here the data engineers would work with the data scientist to automate the data ingestion through data engineering tools. The software engineers would work with data scientists to run integration tests for the models. The computing would be managed and experiment results will be tracked through tools like mlflow. The code would be version controlled. The testing scripts would also be automated. The release would still be manual

Level 3: Automated Model Deployment

This contains everything similar to Level when it comes to data ingestion and model training. The model release however will be automated through Ci/CD pipelines. The testing and scoring would be done through tests like A/B tests. The model performances would also be tracked through tools like Model Registry

Level 4:Full MLOps Automated Retraining

The full system from data ingestion to model deployment and maintenance is automated and monitored. The model retraining and testing would also likely be automated through the code. This aims at achieving a zero down time system

In conclusion, the MLOps maturity model is used to determine the tools and practices which would be used for different use cases. For a POC-like use case most likely the Level 0 model can be used. For a fully-fledged recommendation system level 4 can be used. For an internal platform level 3 or level 2 can be used.