Overview of MLOps with Open Source Tools

Introduction to MLOps

Machine Learning Operations, or MLOps for short, is an emerging practice for collaboration and communication between data scientists and operations professionals to help manage the production machine learning lifecycle. Similar to how DevOps revolutionized development by breaking down silos between software development and operations teams, MLOps aims to bring together data scientists, who develop machine learning models, with operations teams responsible for deploying and monitoring those models in production.

Deploying and managing machine learning models in production presents some unique challenges compared to traditional software. ML models are built using experimental techniques, trained on ever-evolving datasets, and can have nondeterministic behavior influenced by random seeds and hyperparameters. Models need to be continuously monitored for data drift and retrained to maintain performance. Reproducibility is difficult when different teams and individuals work on different parts of the data and ML pipeline.

This is where MLOps comes in – it provides a set of practices and tools to address these challenges. By treating machine learning models as "code", MLOps applies DevOps principles like source control, continuous integration and deployment (CI/CD), and monitoring to the ML development lifecycle. This allows data science and operations teams to collaborate more effectively, deploy models faster and more frequently, and ensure the reliability of models running in production.

The benefits of adopting MLOps include:

• Faster time-to-market by automating the build, test, and deployment of models
• Improved reliability through continuous monitoring and validation of model performance
• Better reproducibility and governance through versioning of code, data, and models
• Increased collaboration between data scientists and engineering teams

Open Source Tools for MLOps

There are a growing number of open source tools and platforms that support MLOps at every stage of the machine learning lifecycle. Here are some of the key categories and popular tools in each:

Experiment Tracking and Model Versioning

Experiment tracking involves recording and managing the different parameters, code versions, and results for each training run. Model versioning means keeping track of the various versions of models created during development. Tools in this category include:

• MLflow: Developed by Databricks, MLflow is an open source platform for managing the ML lifecycle, including tracking experiments, packaging code into reproducible runs, and sharing and deploying models. It has APIs and SDKs for common ML frameworks like PyTorch, TensorFlow and scikit-learn.

• Data Version Control (DVC): DVC is a version control system for machine learning projects. It allows data scientists to version control their datasets and models, making it easy to track changes, switch between versions, and reproduce past experiments. DVC also has a built-in experiment tracking system.

Workflow Orchestration

Workflow orchestration tools allow you to stitch together the various steps in an ML pipeline, from data ingestion and preparation to model training and deployment. They enable automation, scheduling, and monitoring of workflows. Popular tools include:

• Apache Airflow: Originally developed at Airbnb, Apache Airflow is a platform to programmatically author, schedule and monitor workflows as directed acyclic graphs (DAGs) of tasks. It has a large ecosystem of plugins and integrations with data and ML tools.

• Kubeflow Pipelines: Kubeflow is a machine learning toolkit for Kubernetes. Kubeflow Pipelines are a component of Kubeflow that enable the creation and deployment of portable, scalable ML workflows on Kubernetes using a Python SDK and YAML definitions.

Model Deployment and Serving

Model deployment is the process of putting a trained model into production so that it can be used to make predictions on live data. Model serving refers to the infrastructure for hosting models and exposing them via APIs for applications to consume. Tools in this space include:

• KFServing: KFServing is a serverless inferencing platform built on top of Knative and Kubernetes that enables deploying and serving ML models using frameworks like TensorFlow, PyTorch, scikit-learn, and more. It supports canary rollouts, explainability, and payload logging.

• Seldon Core: Seldon Core is an open source platform for deploying machine learning models on Kubernetes. It supports multiple frameworks, languages and deployment strategies like canary rollouts and multi-armed bandits. It also includes analytics and monitoring.

• BentoML: BentoML is an open source framework for building, shipping and running machine learning services. It packages trained models into self-contained Docker images, creates APIs automatically, and supports deployment to platforms like AWS Lambda and Kubernetes.

Feature Stores

A feature store is a central place to store curated features for use in machine learning models. It acts as a bridge between raw data and models. Open source feature stores include:

• Feast: Feast is an open source feature store for machine learning. It allows teams to define, store, and serve machine learning features to models in production, reducing the complexity of data pipelines and promoting reuse of features across models.

• Hopsworks Feature Store: The Hopsworks Feature Store is a data platform for machine learning that provides tools for data engineering and a feature store for building and sharing features across teams and models. It includes capabilities for feature logging, transformation, and serving.

ML Metadata Stores

Metadata stores are a central place to collect and query metadata generated during ML workflows. By recording details on datasets, jobs, models and evaluation metrics, ML metadata stores help govern, discover and reproduce model artifacts. A leading example is:

• ML Metadata: ML Metadata (MLMD) is a library for recording and retrieving metadata associated with ML developer and data scientist workflows. It has APIs for most ML frameworks and can be used to track datasets, jobs, models and evaluation metrics. MLMD can be used on its own or integrated with orchestrators like Kubeflow Pipelines and TFX.

Hands-on Examples

Here are some hands-on tutorials demonstrating how to use open source MLOps tools:

Setting up MLflow for Experiment Tracking

1. Install MLflow: pip install mlflow
2. Import MLflow and log parameters, metrics, and models:

import mlflow

# Log parameters
mlflow.log_param("alpha", 0.5)

# Log metrics
mlflow.log_metric("accuracy", 0.9)

# Log model
mlflow.sklearn.log_model(model, "model")

3. View experiment runs and compare in the MLflow UI:
   mlflow ui

Creating a CI/CD pipeline with GitHub Actions and Kubeflow

1. Create a GitHub repository with your Kubeflow pipeline definition in a file named ml-pipeline.py

2. Set up a GitHub Actions workflow in .github/workflows/ml-pipeline.yml:

name: Kubeflow Pipeline
on: [push]

jobs:
  run-kubeflow-pipeline:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2

    - name: Set up Python
      uses: actions/setup-python@v2

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install kfp

    - name: Compile pipeline
      run: dsl-compile --py ml-pipeline.py --output ml-pipeline.tar.gz

    - name: Deploy pipeline
      env:
        KFP_HOST: ${{ secrets.KFP_HOST }}
      run: kfp --endpoint $KFP_HOST pipeline upload -p demo-pipeline ml-pipeline.tar.gz

This workflow compiles the Kubeflow pipeline and deploys it to a Kubeflow cluster whenever code is pushed. You will need to set the KFP_HOST secret to the URL of your Kubeflow deployment.

Deploying Models with BentoML

1. Train a model using your preferred ML framework

2. Import BentoML and save the model:

import bentoml

# Save model 
saved_model = bentoml.sklearn.save_model("my_model", model)

3. Write a prediction service:

from bentoml.io import NumpyNdarray

iris_clf_runner = bentoml.sklearn.get("my_model:latest").to_runner()

svc = bentoml.Service("iris_classifier", runners=[iris_clf_runner])

@svc.api(input=NumpyNdarray(), output=NumpyNdarray())
def classify(input_series: np.ndarray) -> np.ndarray:
    result = iris_clf_runner.predict.run(input_series)
    return result

4. Build a Bento:
   bentoml build

5. Containerize model API server:
   bentoml containerize iris_classifier:latest

6. Deploy to cloud platforms like AWS Lambda or Kubernetes

Best Practices for MLOps

To succeed with MLOps, you should adopt these key best practices:

• Version control everything: Version your code, datasets, models and experiments using tools like Git and DVC. This makes it possible to reproduce past results and track what has changed between runs.

• Automate testing and validation: Include automated tests and validation checks in your CI/CD pipelines to catch errors early and ensure model quality before deployment. Validate model inputs and outputs, test model performance on holdout data.

• Optimize for reproducibility: Package your code, models and dependencies together so that others can reliably reproduce your work. Use containers or virtual environments to isolate dependencies. Adopt standard conventions for filesystem layout and naming.

• Monitor models in production: Don‘t assume your models will continue to perform well on live data. Use monitoring tools to detect data drift, model decay, and other issues. Set up alerts to proactively surface problems. Have a plan for rolling back or updating models if needed.

• Govern access and changes: Implement controls and approval workflows to govern how data and models are accessed and changed. Maintain an audit trail of who did what when. Regularly review permissions and access logs.

Future of MLOps

As an emerging practice, MLOps will continue to evolve with new tools and methodologies. Some key trends and areas to watch include:

• Convergence with DataOps and DevOps: MLOps will increasingly overlap and integrate with data engineering (DataOps) and software engineering (DevOps) workflows, enabling true end-to-end automation of the data and ML lifecycle.

• Increased focus on governance and security: As companies deploy more mission-critical ML models, there will be a greater emphasis on governance, risk, and compliance. Look for more tools to help with access controls, audit trails, and regulatory requirements.

• Advances in AI explainability and fairness: There is growing interest in techniques to interpret black-box ML models and ensure they are making fair, unbiased predictions. Expect to see more MLOps tools incorporate explainable AI and algorithmic fairness capabilities.

• Serverless deployment of models: Deploying and scaling machine learning models is still challenging, especially for real-time serving. The rise of serverless computing will make it easier to deploy ML models without having to manage infrastructure.

• MLOps for edge devices: As machine learning moves to edge devices like mobile phones and IoT sensors, we will need new MLOps practices and tools to manage the unique challenges of deploying and monitoring ML models on constrained, remote devices.

Conclusion

MLOps is a critical practice for organizations that want to successfully deploy machine learning models into production. By adopting DevOps best practices for machine learning and leveraging open source MLOps tools across the lifecycle, data scientists and operations teams can collaborate better to build, deploy and maintain high-quality ML-powered applications.

There is a rich and growing ecosystem of open source tools to help implement MLOps, including MLflow and DVC for experiment tracking and model versioning, Kubeflow Pipelines and Apache Airflow for workflow orchestration, BentoML and KFServing for model deployment and serving, and Feast for feature storage. By adopting these tools and following best practices like versioning everything, automating testing, optimizing for reproducibility, and monitoring models in production, organizations can realize the full potential of MLOps.

To get started with MLOps, we recommend identifying a pilot project to test out some of these tools and practices. Start small and simple, document what you learn, and iterate from there. Over time, you can formalize your MLOps processes, create templates and cookbooks, and scale up to more projects and teams.

The most important thing is to just get started applying DevOps principles and tools to your machine learning workflows. The future of ML is dependent on robust, production-ready models, and MLOps is the key to getting there. Stay on top of this fast-moving space and don‘t be afraid to experiment with emerging tools and practices. Before long, you‘ll be operating machine learning at scale!