Building Trust in Machine Learning Models with LIME Explanations in Python

Machine learning models are increasingly being used to make critical decisions in high-stakes domains such as healthcare, finance, and criminal justice. As the saying goes, with great power comes great responsibility. If we are relying on ML models to drive important real-world actions, it‘s crucial that we can trust their predictions and understand how they arrive at them.

However, there is often a trade-off between model accuracy and interpretability. The most powerful state-of-the-art models like deep neural networks and ensembles tend to be complex "black boxes". Their internal logic is opaque and inscrutable to human inspection. Simpler models like linear regression and decision trees are easier to interpret, but usually can‘t match the predictive accuracy of more sophisticated approaches.

Historically, the emphasis in much of the ML community has been on maximizing model performance metrics, even at the cost of interpretability. But in order for ML to be responsibly deployed in the real world, we need to strike a better balance. We need tools and techniques to pry open the black box and shine a light on how models make their predictions.

This is where a technique called Local Interpretable Model-Agnostic Explanations (LIME) comes in. LIME is a method for generating explanations of how any machine learning model makes its predictions. It was introduced in the 2016 paper "Why Should I Trust You?": Explaining the Predictions of Any Classifier and has become popular for its flexibility and ease of use.

How LIME Works

At a high level, LIME generates explanations by approximating a complex model with a simpler, more interpretable model in the local neighborhood around a particular prediction. It does this by generating a new dataset of perturbed samples around the instance to be explained and seeing how the model‘s predictions change. LIME then fits an interpretable model, like a linear regression or decision tree, to this locally-generated dataset.

The key idea is that even if the original model is highly nonlinear and uninterpretable globally, its behavior near a particular instance can be well-approximated by a simpler model. The weights of this local linear model, or the structure of a local decision tree, then serve as an explanation for what features drove this particular prediction.

A nice property of LIME is that it is model-agnostic – it can be applied to any model that can be queried for predictions. It doesn‘t need to access the internals or gradients of the original model. This makes it very flexible and easy to use across different model frameworks.

The LIME paper and the associated Python library provide support for explaining models for tabular data, text, and images. Here we‘ll focus on using LIME for tabular data, but the core concepts are similar across modalities.

Hands-on with LIME in Python

To illustrate how LIME works in practice, let‘s walk through an end-to-end example using Python and a public dataset. We‘ll use the Cat in the Dat dataset from a Kaggle competition. This is a binary classification task to predict whether a customer will default on their payments based on various categorical and numeric features.

First, we‘ll load the data and do some basic preprocessing and train/test splitting:

import pandas as pd
from sklearn.model_selection import train_test_split

data = pd.read_csv(‘cat-in-the-dat/train.csv‘)
X = data.drop([‘target‘], axis=1)
y = data[‘target‘]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Next, let‘s train a few different models on this data and evaluate their accuracy:

from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from xgboost import XGBClassifier
from sklearn.metrics import accuracy_score

models = [
    LogisticRegression(max_iter=1000),
    RandomForestClassifier(),
    XGBClassifier()
]

for model in models:
    model.fit(X_train, y_train)
    preds = model.predict(X_test)
    print(f"{model.__class__.__name__} accuracy: {accuracy_score(y_test, preds):.3f}")

This prints:

LogisticRegression accuracy: 0.794
RandomForestClassifier accuracy: 0.818
XGBClassifier accuracy: 0.824

We can see that all the models perform reasonably well, with the XGBoost model having the highest accuracy. But what are these models actually basing their predictions on? This is where LIME comes in.

Let‘s install the LIME library and import the relevant functions:

!pip install lime
import lime
import lime.lime_tabular

Now let‘s instantiate a LIME explainer object:

categorical_features = [‘bin_0‘, ‘bin_1‘, ‘bin_2‘, ‘bin_3‘, ‘bin_4‘, ‘nom_0‘, ‘nom_1‘, ‘nom_2‘, ‘nom_3‘, ‘nom_4‘, ‘nom_5‘, ‘nom_6‘, ‘nom_7‘, ‘nom_8‘, ‘nom_9‘, ‘ord_0‘, ‘ord_1‘, ‘ord_2‘, ‘ord_3‘, ‘ord_4‘, ‘ord_5‘, ‘day‘, ‘month‘]

explainer = lime.lime_tabular.LimeTabularExplainer(
    training_data=X_train.values,
    mode=‘classification‘, 
    feature_names=X_train.columns.tolist(),
    class_names=[‘no default‘, ‘default‘],
    categorical_features=[X_train.columns.get_loc(col) for col in categorical_features if col in X_train.columns],
    discretize_continuous=True
)

Some key parameters to the LimeTabularExplainer:

• training_data: The original training data the model was trained on. LIME uses this to generate realistic perturbed samples.
• mode: ‘classification‘ or ‘regression‘ depending on the task
• feature_names: The names of the features, so they can be displayed in the explanation
• class_names: The names of the target classes in a classification task
• categorical_features: A list of indices of categorical features. This lets LIME perturb them in a way that respects their categorical nature.
• discretize_continuous: Whether to discretize continuous features into quartiles. This can make explanations more interpretable.

Next, we define some helper functions to get predictions from each model:

def rf_predict_proba(X):
    return model_rf.predict_proba(X)

def lr_predict_proba(X):
    return model_lr.predict_proba(X)

def xgb_predict_proba(X):
    return model_xgb.predict_proba(X)

Now we‘re ready to generate some explanations! Let‘s explain the predictions for the first few instances in the test set for each model.

for i in range(3):
    print(f"Instance {i}:")
    print(f"Actual label: {y_test.iloc[i]}")

    exp_rf = explainer.explain_instance(X_test.iloc[i], rf_predict_proba, num_features=5)
    print("Random Forest explanation:")
    exp_rf.show_in_notebook(show_all=False)

    exp_lr = explainer.explain_instance(X_test.iloc[i], lr_predict_proba, num_features=5)
    print("Logistic Regression explanation:")
    exp_lr.show_in_notebook(show_all=False)

    exp_xgb = explainer.explain_instance(X_test.iloc[i], xgb_predict_proba, num_features=5)
    print("XGBoost explanation:")
    exp_xgb.show_in_notebook(show_all=False)

    print("-----")

This generates explanations like:

Each explanation shows the top features contributing to the prediction for this instance, along with their weights. Green features push the prediction towards the positive class (default), while red features push it towards the negative class (no default). The explanation also shows the actual values of these features for this data point.

We can see that different models are picking up on different features as important. The logistic regression is putting a lot of weight on the nom_7 feature, while the random forest and XGBoost models are spreading weight more evenly across multiple features.

It‘s important to remember that these are local explanations, specific to each individual prediction. The patterns and important features could be quite different for another instance. This is the power of LIME – it lets us zoom in and understand what drove a particular prediction, rather than just providing global feature importances.

Best Practices and Caveats

While LIME is a powerful tool for model interpretation, there are a few things to keep in mind when using it:

1. Choosing the local model: LIME supports using different types of local models to generate explanations, like linear models, decision trees, or rule lists. The choice of model affects the form and interpretability of the explanation. Linear models provide feature weights, while decision trees show a logical decision path. It‘s worth experimenting with different local models to see what provides the most insightful explanations for your use case.

2. Handling categorical features: LIME provides support for correctly perturbing categorical features, but it‘s important to specify which features are categorical so they are treated appropriately. Incorrectly treating a categorical feature as numeric can lead to unrealistic perturbed samples and misleading explanations.

3. Validating explanations: It‘s important to spot check the explanations generated by LIME to ensure they match domain knowledge and intuition. If the important features highlighted by LIME don‘t make sense, it could be a sign of a problem with the model or the data. LIME is a tool for understanding models, but it‘s not a substitute for careful validation.

4. Combining with global methods: LIME provides local explanations for individual predictions, but it‘s also useful to understand the global behavior of a model. Global feature importance methods like permutation importance can provide a complementary view. Using both local and global explanation techniques can give a more complete picture.

5. Considering the audience: The purpose of LIME explanations is to provide insight and build trust with stakeholders. It‘s important to consider who will be consuming the explanations and tailor them accordingly. A machine learning engineer may want to see all the technical details, while a business user may prefer a more high-level summary. LIME provides flexibility in how explanations are displayed and visualized.

The Future of Interpretable Machine Learning

LIME is just one of many techniques in the rapidly growing field of interpretable machine learning and explainable AI (XAI). Other approaches include:

• Rule extraction: Methods like anchors and decision rule lists that provide explanations in the form of logical rules that approximate a model‘s behavior.

• Concept activation vectors: Techniques for interpreting deep neural networks by identifying high-level human-understandable concepts that different neurons or layers respond to.

• Counterfactual explanations: Explanations that show how an instance would need to change in order to flip the model‘s prediction. These can be more actionable than just listing important features.

• Inherently interpretable models: An emerging area of research is developing models that are inherently interpretable by design, such as generalized additive models, Bayesian rule lists, and deep symbolic models. These aim to provide the best of both worlds – the performance of sophisticated models with the interpretability of simpler ones.

An important trend is the recognition that explanations need to be tailored to the particular audience and use case. An explanation that satisfies a model developer may not be suitable for an end user or regulator. The field is moving towards more human-centered explanations that take into account the cognitive biases, mental models, and needs of the consumer of the explanation.

As machine learning is deployed in increasingly high-stakes domains, being able to explain and justify model predictions is becoming a necessity rather than an afterthought. Interpretability is likely to become a key evaluation criterion alongside traditional metrics like accuracy and AUC.

This means that model developers and data scientists need to have interpretability techniques like LIME in their toolbox. They should proactively surface model explanations rather than waiting to be asked. By doing so, they can build trust and confidence in their models, catch potential issues early, and make their work more impactful.

Conclusion

We‘ve seen how LIME can be a powerful tool for peeking inside the black box of complex machine learning models. By providing local explanations for individual predictions, LIME helps build trust and understanding of how models behave.

However, LIME is just one piece of the puzzle. Truly interpretable machine learning requires a holistic approach combining multiple explanation techniques, careful validation, and a consideration for the needs of the explanation consumer.

As the field matures, we can expect to see interpretability become a key pillar of responsible AI alongside other areas like fairness, privacy, and security. Machine learning practitioners who embrace interpretability will be well-positioned to build high-performing, responsible, and trustworthy models.

To learn more about interpretable machine learning, check out these resources:

• The LIME paper: "Why Should I Trust You?": Explaining the Predictions of Any Classifier
• The Interpretable Machine Learning Book by Christoph Molnar: https://christophm.github.io/interpretable-ml-book/
• The XAI Stories interactive demos: https://pbiecek.github.io/xai_stories/

Here‘s to more interpretable, trustworthy, and responsible machine learning!