Building an Interactive Classification Model Simulator with Dash and Python

Machine learning models for classification problems are widely used in many domains from finance to healthcare. Data scientists often need to rapidly prototype classification models and interactively tune parameters to see how they impact model performance. Dash is an open source framework for building analytical web applications in Python that is ideal for this use case.

In this article, we‘ll walk through how to use Dash to build an interactive application for simulating different classification models. The app will allow the user to select a dataset, pick the target and feature variables, choose a model type, set model hyperparameters, and visualize the results. We‘ll cover the end-to-end process from loading the data to displaying model evaluation metrics.

By the end, you‘ll see how Dash allows you to integrate the power of Python data science libraries with an intuitive user interface to build compelling data apps. Let‘s dive in!

What is Dash?

Dash is a productive framework for building web applications in Python. It‘s especially well-suited for data science and machine learning apps that need to be interactive and visually appealing. Dash apps are composed of two parts:

1. The layout that describes what the application looks like
2. The callbacks that describe the interactivity

The layout is composed of standard HTML components like buttons, dropdowns, sliders as well as graph components for displaying interactive plots. Dash provides a simple Python interface to these components so you can add them to your application without having to write HTML or JavaScript.

The callbacks are functions that are automatically called by Dash whenever an input component‘s property changes. Callbacks allow the input components to interact with each other and with your data model.

App Components

Our classification model simulator app will have the following main components:

1. Dataset selection dropdown
2. Feature selection dropdown
3. Target variable selection dropdown
4. Train/test split slider
5. Model selection dropdown
6. Model hyperparameter inputs
7. Train model button
8. Model performance metrics display
9. Interactive model evaluation plots

We‘ll use a combination of Dash core components, Dash DAQ components, and Plotly graphs to build this interface.

Loading the Dataset

First we‘ll load our training dataset into a Pandas dataframe. For this example, we‘ll use the Adult Census Income dataset from Kaggle. This dataset has features like age, workclass, education, marital status, etc. and the target is a binary variable for whether the person‘s income exceeds $50K per year.

import pandas as pd

data = pd.read_csv("adult.csv")
print(data.head())

This prints the first few rows:

   age         workclass  fnlwgt  education  education-num      marital-status         occupation   relationship   race     sex  capital-gain  capital-loss  hours-per-week native-country salary
0   39         State-gov   77516  Bachelors             13       Never-married       Adm-clerical  Not-in-family  White    Male          2174             0              40  United-States  <=50K
1   50  Self-emp-not-inc   83311  Bachelors             13  Married-civ-spouse    Exec-managerial        Husband  White    Male             0             0              13  United-States  <=50K
2   38           Private  215646    HS-grad              9            Divorced  Handlers-cleaners  Not-in-family  White    Male             0             0              40  United-States  <=50K
3   53           Private  234721       11th              7  Married-civ-spouse  Handlers-cleaners        Husband  Black    Male             0             0              40  United-States  <=50K
4   28           Private  338409  Bachelors             13  Married-civ-spouse     Prof-specialty           Wife  Black  Female             0             0              40           Cuba  <=50K

User Interface

Now let‘s design our user interface in Dash. We‘ll use a two-column layout with the left column containing the input controls and model metrics, and the right column containing the plots.

import dash_core_components as dcc
import dash_html_components as html
from dash.dependencies import Input, Output

# Left column
left_col = html.Div([
    html.Label("Dataset"),
    dcc.Dropdown(
        id="dataset-dropdown",
        options=[{"label": d, "value": d} for d in ["Adult Census Income"]],
        value="Adult Census Income"
    ),

    html.Label("Features"),
    dcc.Dropdown(
        id="feature-dropdown",
        options=[{"label": c, "value": c} for c in data.columns],
        value=data.columns[:6],
        multi=True
    ),

    html.Label("Target"),
    dcc.Dropdown(
        id="target-dropdown",
        options=[{"label": c, "value": c} for c in data.columns],
        value="salary"  
    ),

    html.Label("Train/Test Split"),
    dcc.Slider(
        id="split-slider",
        min=0.1,
        max=0.9,
        step=0.1,
        value=0.7,
        marks={i: str(i) for i in [0.1, 0.3, 0.5, 0.7, 0.9]} 
    ),

    html.Label("Model"),
    dcc.Dropdown(
        id="model-dropdown",
        options=[{"label": m, "value": m} for m in ["Logistic Regression", "Random Forest", "Gradient Boosting"]],
        value="Logistic Regression"
    ),

    html.Button("Train Model", id="train-button"),

    html.Div(id="metrics")
])


# Right column  
right_col = html.Div([
    dcc.Graph(id="roc-curve"),
    dcc.Graph(id="confusion-matrix")
])

# Overall layout
app.layout = html.Div([
    html.H1("Classification Model Simulator"),
    html.Div([left_col, right_col]) 
])

This defines the basic layout with dropdowns for selecting dataset, features, target, a slider for the train/test split proportion, a dropdown for model selection and a button to trigger model training.

The empty Div with id metrics will be populated by the model performance metrics that we‘ll calculate in the callback. The two dcc.Graph components in the right column will hold the ROC curve and confusion matrix plots.

Model Training

Next we need to write the callback function that will be triggered when the "Train Model" button is clicked. This function will:

1. Read the selected options from the input components
2. Subset the features and split into train/test sets
3. Train the selected model
4. Calculate model performance metrics
5. Create the ROC curve and confusion matrix figures
6. Return the metrics and figures to update the output components

Here‘s the code for this (we‘ll go through it step-by-step):

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.metrics import roc_curve, confusion_matrix

import plotly.express as px

@app.callback(
    [Output("metrics", "children"),
     Output("roc-curve", "figure"),
     Output("confusion-matrix", "figure")],
    [Input("train-button", "n_clicks")],
    [State("feature-dropdown", "value"),
     State("target-dropdown", "value"),
     State("split-slider", "value"),
     State("model-dropdown", "value")]
)
def train_model(n_clicks, features, target, split, model):
    # Subset data
    X = data[features]
    y = data[target]

    # Split into train/test
    X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=split, random_state=42)

    # Fit model
    if model == "Logistic Regression":
        clf = LogisticRegression()
    elif model == "Random Forest":
        clf = RandomForestClassifier()
    else:
        clf = GradientBoostingClassifier()

    clf.fit(X_train, y_train)

    # Predict
    y_prob = clf.predict_proba(X_test)[:,1]
    y_pred = clf.predict(X_test)

    # Calculate metrics  
    fpr, tpr, thresholds = roc_curve(y_test, y_prob)
    cm = confusion_matrix(y_test, y_pred)

    accuracy = round(clf.score(X_test, y_test), 3)
    precision = round(cm[1,1] / (cm[0,1] + cm[1,1]), 3) 
    recall = round(cm[1,1] / (cm[1,0]+ cm[1,1]), 3)

    metrics = [
        html.H3(f"Accuracy: {accuracy}"),  
        html.H3(f"Precision: {precision}"),
        html.H3(f"Recall: {recall}")
    ] 

    # Create figures
    roc_fig = px.area(
        x=fpr, y=tpr,
        title=f‘ROC Curve (AUC={metrics.auc(fpr, tpr):.4f})‘,
        labels=dict(x=‘False Positive Rate‘, y=‘True Positive Rate‘),
        width=500, height=400
    )

    roc_fig.add_shape(
        type=‘line‘, line=dict(dash=‘dash‘),
        x0=0, x1=1, y0=0, y1=1
    )

    roc_fig.update_yaxes(scaleanchor="x", scaleratio=1)
    roc_fig.update_xaxes(constrain=‘domain‘)

    cm_fig = px.imshow(
        cm, 
        x=[‘Predicted 0‘, ‘Predicted 1‘],
        y=[‘Actual 0‘, ‘Actual 1‘], 
        labels=dict(x="Predicted", y="Actual"),
        text_auto=True, color_continuous_scale=‘Blues‘
    )

    return metrics, roc_fig, cm_fig

Let‘s break this down:

The @app.callback decorator specifies that this function should be called whenever the "Train Model" button is clicked (n_clicks property changes). The function also depends on the current state of the feature and target dropdowns, split slider, and model dropdown, so we pass those in using the State function.

Inside the function, we first subset the dataframe to get the selected features X and target y. Then we split into train and test sets using the train_test_split function from scikit-learn, passing in the selected split proportion.

Next we initialize the selected model class and fit it on the training data. To get the predicted probabilities for the positive class, we call clf.predict_proba(X_test)[:,1], and to get the class predictions we use clf.predict(X_test).

To calculate the ROC curve and confusion matrix, we use the roc_curve and confusion_matrix functions from scikit-learn. We also calculate some common classifier metrics like accuracy, precision and recall.

Finally, we create the ROC curve and confusion matrix plots using the px.area and px.imshow functions from the Plotly Express library. We customize the figures‘ titles, labels, size, and color scale to make them look nice.

The function returns the list of metric elements to populate the metrics Div, and the two figures to update the roc-curve and confusion-matrix graphs.

Enhancements

There are a number of enhancements we could make to this app:

• Error logging with the Python logging module to track any exceptions that occur
• Saving the trained models and metrics to a database or file
• Adding more plot types like precision-recall curves, feature importances, etc.
• Allowing the user to tune model hyperparameters for more control
• Writing PyTest unit tests to verify the callbacks

Conclusion

In this article, we saw how to use the Dash framework to build an interactive web application for training and evaluating classification models. Dash makes it simple to assemble a UI from components, hook it up to your Python model code, and deploy it as a standalone app.

The app we built allows the user to interactively select a dataset, features, train/test split, and model type, and see the resulting model performance metrics and plots. This kind of rapid iteration is a huge productivity boost for data scientists who need to experiment with many models.

I hope this tutorial has shown you the power and flexibility of the Dash framework for building data science web apps. The complete source code is available on my GitHub. Feel free to use it as a starting point for your own Dash apps.

Thanks for reading! Let me know if you have any questions or suggestions.