6 Python Libraries to Interpret Machine Learning Models and Build Trust

Why Do We Need Interpretable Machine Learning Models?

Machine learning and deep learning models have become incredibly sophisticated and powerful in recent years. From computer vision to natural language processing, ML models are achieving unprecedented performance across a variety of domains and revolutionizing entire industries.

However, with greater complexity and abstraction, these models also run the risk of becoming inscrutable "black boxes". It can be difficult to explain how the models arrive at their predictions, even for the data scientists and ML engineers who build them.

This is a problem, because in order for machine learning to be successfully integrated into real-world applications, the models need to be trusted by the humans who use them. Think about high-stakes domains like healthcare, finance, autonomous driving, criminal justice and so on – the costs of mistakes can be huge, and blind faith in complex ML models is simply not feasible.

Imagine a doctor trying to diagnose a patient‘s disease. They feed the patient‘s data into a state-of-the-art deep learning model, and it predicts a rare, life-threatening condition with 95% confidence. Can the doctor simply trust the model‘s prediction and start administering an aggressive treatment regime? Probably not – they would want to understand why the model made that prediction, look at the key factors that led to the prediction, and combine it with their own clinical judgment before proceeding.

Similarly, if a bank uses an ML model to decide on loan approvals, and it rejects a loan application, they need to be able to point to the specific factors that contributed to that decision. "The model said so" is not an appropriate explanation! The customer deserves to know the rationale behind the rejection.

This is where interpretable machine learning comes in. The goal is to open up the black box and shed light on the inner workings of ML models, so that their predictions can be understood and trusted by human users. In this article, we‘ll explore six powerful Python libraries that can help you build more interpretable ML models.

Let‘s dive in!

1. ELI5

ELI5, which stands for "Explain Like I‘m 5", is a Python library that aims to simplify and visualize machine learning models. It provides both global and local interpretations of models.

Global interpretations look at the model‘s parameters and weights to explain how the model works as a whole. ELI5 provides a show_weights() function that displays the model‘s feature weights, giving you a sense of which features are most important to the model‘s predictions.

Local interpretations, on the other hand, focus on individual predictions and identify the specific features that led to that prediction. ELI5‘s show_prediction() function takes in an individual data instance and explains the factors behind the model‘s prediction for that instance.

One of the best things about ELI5 is that it supports many popular ML libraries out-of-the-box, including scikit-learn, XGBoost, Keras, and more. So you can easily integrate it with your existing ML workflow.

ELI5 also makes it easy to communicate your model interpretations to non-technical stakeholders. It can generate explanations in HTML, JSON or even pandas DataFrame format, which you can then embed into webpages, APIs or reports.

Finally, ELI5 has a special TextExplainer module that is designed for interpreting text classification models. So if you‘re working on an NLP problem, give ELI5 a shot!

2. LIME

LIME, or Local Interpretable Model-Agnostic Explanations, is a technique that explains individual predictions of any ML model in an interpretable and faithful manner. It was developed by researchers at the University of Washington.

The key idea behind LIME is to perturb the input around a specific data instance and see how the predictions change. By doing this repeatedly, LIME identifies the features that have the greatest impact on the prediction for that instance.

For example, let‘s say you have a model that predicts whether a movie review is positive or negative. For a specific review that the model predicts as negative, LIME would identify the specific words and phrases in the review that pushed the model towards the negative prediction.

LIME satisfies several important criteria for explanations:

1. Interpretable: The explanations need to be understandable to the intended audience (e.g. doctors, loan officers, end users)
2. Local fidelity: The explanations should faithfully represent how the model behaves for that specific input, even if it doesn‘t explain the model globally
3. Model-agnostic: The technique should work for any ML model
4. Global perspective: The explanations should provide insight about the model‘s overall behavior

The LIME Python library supports explanations for tabular data, text and images. The API is pretty straightforward – you essentially need to provide the trained model‘s prediction function, the training data, and optionally the names of the features and classes.

There are also several variations of LIME that have been proposed, such as aLIME (for explaining Recurrent Neural Nets), DLIME (for Deep Neural Nets) and SHAP (SHapley Additive exPlanations), which we‘ll cover next.

3. SHAP

SHAP, which stands for SHapley Additive exPlanations, is a technique to explain ML models based on the game theoretic concept of Shapley values. It was developed by researchers at the University of Washington.

In game theory, Shapley values are a way to fairly distribute the "payout" among the "players" in a cooperative game. In the context of ML models, SHAP interprets the features as "players" and the prediction as the "payout". It then computes Shapley values, which indicate how to fairly distribute the prediction among the input features.

Intuitively, a feature‘s Shapley value represents the feature‘s contribution to the model‘s prediction, while taking into account the interaction effects between features. A positive Shapley value means the feature pushed the prediction higher, while a negative Shapley value means the feature pushed the prediction lower.

One of the key advantages of SHAP is that the explanations generated are additive. This means that the Shapley values for all the features sum up to the final model prediction. So you get a full account of the model‘s behavior.

The SHAP Python library provides several different explainer algorithms, each suited for different types of models and data. For tree-based models like Decision Trees and Random Forests, SHAP offers a fast and exact Tree Explainer. There‘s also a Kernel Explainer that works for any model, but is slower.

To use SHAP, you first initialize the appropriate explainer with your trained model. Then you call the shap_values() function, passing in the data instances you want to explain. This returns a matrix of Shapley values, which you can then visualize in various ways using SHAP‘s plotting functions.

SHAP also offers some really nice visualizations, like force plots (which show how each feature value contributes to the prediction), summary plots (which show the overall importance of each feature), and dependence plots (which show how a feature‘s impact varies across its range).

4. Yellowbrick

Yellowbrick is a Python library that provides a suite of visual analysis and diagnostic tools for ML models. It‘s built on top of scikit-learn and matplotlib, so if you‘re already familiar with those libraries, you‘ll feel right at home with Yellowbrick.

The key concept in Yellowbrick is that of "Visualizers". A Visualizer is a Python class that encapsulates a specific visual analysis or diagnostic tool. To use a Visualizer, you instantiate it with the appropriate parameters, call its fit() and transform() methods on your data and model, and then call its show() method to render the visualization.

This workflow will be very familiar if you‘ve used scikit-learn‘s Estimator and Transformer classes before. In fact, many of Yellowbrick‘s Visualizers wrap scikit-learn‘s models and can be dropped into existing scikit-learn Pipelines.

Some of the key Visualizers offered by Yellowbrick include:

• FeatureImportances: Visualizes the importance of each feature to a model‘s predictions
• PredictionError: Plots the actual vs predicted values for regression models
• ClassificationReport: Provides a visual classification report with precision, recall, F1 score etc.
• ROCAUC: Plots the Receiver Operating Characteristic (ROC) curve and computes the Area Under the Curve (AUC)
• ConfusionMatrix: Plots a visual confusion matrix with colormapped cells
• ClassBalance: Displays the balance of classes in the dataset

There are also Visualizers for clustering, text analysis, feature analysis and selection, and more. The Yellowbrick documentation provides a great overview of all the available Visualizers.

One thing to note is that Yellowbrick‘s Visualizers are mainly focused on global model interpretability, rather than local explanations for individual predictions. But they offer an excellent toolkit for understanding and debugging your ML models during the development process.

5. Alibi

Alibi is an open source Python library for ML model inspection and interpretation. It was developed by Seldon, a company that offers an MLOps platform for deploying and monitoring models in production.

Like LIME and SHAP, Alibi provides instance-level explanations of model predictions. But it offers a wider range of explanation methods, each tailored for different types of data (numerical tabular data, text, images).

For example, Alibi provides an Anchor Explainer, which identifies a set of rules or "anchors" that guarantee a certain prediction. An anchor rule for a text classifier might be something like "IF the review contains the words ‘great‘ and ‘acting‘, THEN the model predicts positive sentiment".

Alibi also offers a Contrastive Explainer, which generates pertinent negatives (inputs that are minimally different from the original instance but lead to a different prediction) and pertinent positives (inputs that are minimally different but lead to the same prediction). This helps users understand the decision boundary of the model.

For neural networks and other black-box models, Alibi provides Integrated Gradients and GradientSHAP Explainers. These methods attribute the prediction to the input features by tracking the gradients through the model.

One of the key advantages of Alibi is that it is model-agnostic – you don‘t need access to the model internals, just the ability to get predictions from the model. This makes it very versatile and easy to integrate into existing ML workflows.

To use Alibi, you first select the appropriate Explainer class for your model and data type. Then you instantiate the Explainer with the model‘s prediction function, and any other required arguments. Finally, you call the explain() method on one or more data instances to generate the explanations.

Alibi also has a built-in saving module that allows you to save the generated explanations to disk, so you can share them with others or use them for compliance and auditing purposes.

6. Lucid

Lucid is a Python library that focuses specifically on understanding and visualizing deep neural networks (DNNs). It was developed by researchers at Google Brain.

DNNs have achieved remarkable performance on many tasks, but their complexity and opacity make them particularly challenging to interpret. Lucid aims to shed light on these models by providing a set of tools to visualize the activations and gradients of DNNs.

One of the key features of Lucid is its optvis (optimization-based visualization) module. This allows you to generate synthetic inputs that maximize the activation of a specific neuron, channel or layer in the network. By looking at these generated inputs, you can get a sense of what the network is "looking for" at that level.

For example, if you optimize for a single neuron in one of the later layers of an image classification network, you might see a ghostly image that resembles the object class that neuron is sensitive to (e.g. a car or a dog). This is a powerful way to probe the network‘s learned representations.

Lucid also provides a modelzoo module with a collection of pretrained models that you can use for visualization. These include popular architectures like Inception, VGG, and ResNet.

To use Lucid, you first import the appropriate model from the modelzoo (or load your own pretrained model). Then you define the objective you want to optimize (e.g. maximizing the activation of a specific neuron), and pass it to the render() function along with the model. This runs the optimization and returns the generated input.

You can also compose multiple objectives to visualize interactions between neurons, channels or layers. And you can apply various transformations (e.g. rotation, scaling) to the input during optimization to generate more diverse and interpretable visualizations.

One thing to note is that Lucid currently only supports TensorFlow models (version 1.x). And while it provides some excellent Colab notebooks to get started, the documentation is still a work in progress. But if you‘re working with DNNs and want to get a deeper understanding of their internals, Lucid is definitely worth checking out.

Conclusion

As ML models become more complex and widespread, model interpretability is becoming increasingly important. We need to be able to explain and justify the predictions of our models, not just to comply with regulations but to build trust with users and stakeholders.

The Python libraries we‘ve covered in this article – ELI5, LIME, SHAP, Yellowbrick, Alibi, and Lucid – offer a range of techniques for interpreting ML models, from global feature importances to instance-level explanations to deep neural network visualizations.

Each library has its own strengths and use cases, so it‘s worth exploring a few of them to see what works best for your particular problem and workflow. And keep in mind that model interpretability is still an active area of research, with new techniques and libraries emerging all the time.

Ultimately, the goal is to build models that are not just accurate, but also transparent, accountable, and trustworthy. By using these interpretability tools, and by making a commitment to open and ethical AI development, we can create machine learning systems that truly benefit society.