Deploying Machine Learning Models with CherryPy and Docker

ByJordan Brown

Developing machine learning models is only half the battle—to provide value, they need to be deployed to production environments where they can be consumed by other systems and end users. Model deployment can be challenging, but leveraging tools like CherryPy and Docker can greatly simplify the process. In this post, we‘ll take an in-depth look at deploying machine learning models as web services using these powerful technologies.

The Machine Learning Model Deployment Process

Deployment is a critical step in the machine learning lifecycle. A model isn‘t useful unless it can be accessed to generate predictions on new data. There are several key phases in deploying a model to a production environment:

  1. Training the model on historical data and evaluating its performance
  2. Saving the trained model artifact
  3. Creating an API or application layer to expose the model
  4. Packaging the model API in a way that can be deployed to the production infrastructure
  5. Deploying and running the packaged model API on a production server or platform
  6. Monitoring the deployed model‘s performance and resource consumption over time

The model deployment process needs to be efficient, repeatable, and scalable. This is where tools like CherryPy and Docker come in. They can help automate and standardize many of these steps.

Introducing CherryPy for Building Python Web APIs

CherryPy is a lightweight, object-oriented web framework for Python. It allows developers to quickly build web applications and APIs by providing key features out-of-the-box:

  • A fast HTTP server to handle requests and responses
  • URL mapping to route requests to handler code
  • Built-in tools for parsing request data and generating responses
  • Session handling and caching support
  • A plugin system for extending functionality

One of the main advantages of CherryPy is its simplicity. A basic web application requires minimal boilerplate code, and the framework enforces a clean separation between business logic and presentation. This makes it an excellent choice for creating APIs to expose machine learning models.

Here‘s what a simple "Hello World" application in CherryPy looks like:

import cherrypy

class HelloWorld(object):
    @cherrypy.expose
    def index(self):
        return "Hello World!"

cherrypy.quickstart(HelloWorld())

When this code is run, it starts a web server and listens for HTTP requests on a default port. Navigating to the application‘s URL in a web browser or making a request using a tool like curl will yield the "Hello World!" response.

Building on this foundation, we can create a web API for serving predictions from a trained machine learning model. We‘ll see a detailed example later in this post.

Packaging and Deploying with Docker Containers

While we can deploy a CherryPy model API directly on a host machine, using containers provides a higher level of portability and consistency. Docker is a platform that simplifies the creation, distribution, and running of applications using containerization.

Containers are lightweight, standalone packages that include an application and all its dependencies. They provide a consistent environment that can be deployed reliably across different host systems. Docker makes it easy to define, build, and run containers.

The key components of the Docker platform include:

  • Dockerfile: A script that defines the contents and configuration of a container image
  • Images: Read-only templates that represent a packaged application, including its code, runtime, libraries, and environment settings
  • Containers: Running instances of an image that are isolated from the host and other containers
  • Docker Engine: The runtime that manages the building and execution of containers
  • Docker Compose: A tool for defining and running multi-container applications
  • Docker Hub: A centralized registry for storing and sharing container images

By packaging a CherryPy model API in a Docker container, we gain several advantages:

  • Environment consistency between development and production
  • Isolation of application dependencies to avoid conflicts
  • Portability across different servers and cloud platforms
  • Scalability through spinning up multiple container instances
  • Streamlined CI/CD workflows using images stored in a registry

We‘ll walk through the process of "Dockerizing" a CherryPy model API in our example.

Example: Deploying a Machine Learning Model with CherryPy and Docker

Now let‘s put the concepts we‘ve covered into practice. We‘ll walk through an end-to-end example of training a machine learning model, creating a CherryPy API to expose it, and deploying the API using Docker.

Step 1: Training the Model

First, we‘ll train a simple logistic regression model on the classic Iris dataset using scikit-learn:

from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

# Load the Iris dataset
iris = load_iris()
X, y = iris.data, iris.target

# Split into train and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y)

# Fit a logistic regression model
model = LogisticRegression()
model.fit(X_train, y_train)

# Evaluate performance on test set
accuracy = model.score(X_test, y_test)
print(f"Test Accuracy: {accuracy:.3f}")

After training the model, we save it to disk using joblib:

from joblib import dump

dump(model, ‘iris_model.pkl‘)

Step 2: Creating the CherryPy Model API

Next, we‘ll create a CherryPy application to serve predictions from our trained model. We‘ll define a /predict endpoint that accepts POST requests containing feature data and returns the model‘s predicted class.

import cherrypy
from joblib import load

class ModelAPI(object):
    def __init__(self):
        self.model = load(‘iris_model.pkl‘)

    @cherrypy.expose
    @cherrypy.tools.json_in()
    @cherrypy.tools.json_out()
    def predict(self):
        # Parse input data from JSON
        data = cherrypy.request.json

        # Convert dict to list of values
        X = [data[f] for f in [‘sepal_length‘, ‘sepal_width‘, 
                               ‘petal_length‘, ‘petal_width‘]]

        # Generate model prediction
        pred_class = self.model.predict([X])[0]

        # Return predicted class as JSON
        return {‘predicted_class‘: pred_class}

if __name__ == ‘__main__‘:
    cherrypy.quickstart(ModelAPI())

This code loads the trained model, defines a /predict endpoint that parses input data from JSON, generates a prediction using the model, and returns the result as a JSON response.

Step 3: Defining the Dockerfile

To run this API in a Docker container, we need to define a Dockerfile that specifies the application‘s environment and dependencies. Here‘s an example Dockerfile:

FROM python:3.8-slim-buster

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY iris_model.pkl .
COPY app.py .

EXPOSE 8080

CMD ["python", "app.py"]

This Dockerfile starts from a base Python image, copies the necessary files into the container, installs the required dependencies from a requirements.txt file, and specifies the command to run the CherryPy application.

Step 4: Building and Running the Docker Container

With the Dockerfile defined, we can build an image and run it as a container using the Docker CLI:

# Build the image 
docker build -t iris-api .

# Run a container from the image
docker run -p 8080:8080 iris-api

The docker build command builds an image tagged iris-api from the current directory, which contains the Dockerfile and application files. The docker run command starts a container from this image, mapping the container‘s port 8080 to the host‘s port 8080.

Step 5: Consuming the Deployed Model API

With the container running, we can now make requests to the model API to generate predictions. Using curl:

curl -X POST http://localhost:8080/predict \
     -H ‘Content-Type: application/json‘ \
     -d ‘{"sepal_length": 5.1, "sepal_width": 3.5,
          "petal_length": 1.4, "petal_width": 0.2}‘

This POST request sends feature data as JSON to the /predict endpoint exposed by the running container. The API returns a JSON response containing the model‘s predicted class for the input data:

{
  "predicted_class": 0
}

We‘ve successfully deployed our machine learning model as a containerized web service! This API can now be consumed by any application that can make HTTP requests.

Best Practices and Considerations for Production Deployment

While our example illustrated the basic process of deploying a model using CherryPy and Docker, there are additional considerations for production environments:

  • Security: Implement authentication, authorization, and secure communication (HTTPS) for your model API. Use Docker secrets to manage sensitive configuration data.

  • Scalability: Use container orchestration tools like Kubernetes to manage running multiple instances of your model API container and load balancing traffic.

  • Monitoring: Implement logging and monitoring for your deployed model to track performance metrics, resource utilization, and errors. Use tools like Prometheus and Grafana for collecting and visualizing this data.

  • Continuous Integration and Deployment (CI/CD): Automate building, testing, and deploying your model API using a CI/CD pipeline. This enables deploying new versions of your model API frequently and reliably.

  • Model Versioning: Use versioned endpoints or a model registry to enable deploying and running multiple versions of your model simultaneously. This allows for easier rollbacks and AB testing.

Alternative Tools and Technologies

While CherryPy and Docker are powerful tools for deploying machine learning models, there are many alternatives worth considering:

  • Web Frameworks: Flask, FastAPI, Django (Python), Plumber (R)
  • Containerization: Podman, Kubernetes, AWS ECS, Azure Container Instances
  • Serverless: AWS Lambda, Google Cloud Functions, Azure Functions
  • Managed ML Platforms: Amazon SageMaker, Google AI Platform, Azure ML, Databricks

The right choice depends on your specific requirements, existing infrastructure, and the skillset of your team.

Conclusion

Deploying machine learning models doesn‘t have to be painful. By leveraging tools like CherryPy for building model APIs and Docker for packaging and running containerized applications, data scientists and ML engineers can quickly and reliably get models into production.

The simple workflow we walked through—training a model, saving it, exposing it through a web API, and deploying it as a container—can be adapted for a wide variety of use cases. While there are additional considerations for running model APIs at scale in production, this foundation can be extended with other best practices and tools as needed.

By making models more accessible and easier to integrate into other systems, data science teams can deliver more value to end users. CherryPy and Docker are valuable additions to the model deployment toolkit.

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

You May Like to Read,

Similar Posts