Using Q-Q Plots to Ensure Your Machine Learning Model is Based on the Right Probability Distributions

As a data scientist building machine learning models, one of the most important steps is understanding the underlying probability distributions of your data. Many common ML algorithms make assumptions about the distributions of the features and target variables they are modeling. If these assumptions are violated, the models may perform poorly or provide misleading results.

Fortunately, there is a powerful tool that can help us visually assess probability distributions and verify assumptions quickly: the Quantile-Quantile or Q-Q plot. In this post, we‘ll take an in-depth look at how Q-Q plots work, how to create them in Python, and how they can be used to select appropriate ML models and validate assumptions.

What are Q-Q Plots?

A Q-Q plot is a graphical method for comparing two probability distributions by plotting their quantiles against each other. In a Q-Q plot, the quantiles (also called percentiles) of one distribution are plotted on the x-axis, and the corresponding quantiles of the second distribution are plotted on the y-axis.

If the two distributions being compared are similar, the plotted points will lie approximately on a straight line. Departures from linearity indicate differences between the distributions. The greater the curvature or deviation from a straight line, the more the distributions diverge.

Q-Q plots get their name from the fact that they compare quantiles. A quantile is a cut point dividing the range of a probability distribution into intervals with equal probabilities. For example, the 0.5 quantile (50th percentile or median) of a distribution is the value such that 50% of the distribution lies below it.

Some common quantiles have special names:

• 0.25 quantile = 25th percentile = first quartile = Q1
• 0.50 quantile = 50th percentile = median = second quartile = Q2
• 0.75 quantile = 75th percentile = third quartile = Q3

In a standard Q-Q plot comparing a sample distribution to a theoretical distribution, the theoretical quantiles are plotted on the x-axis and the sample quantiles on the y-axis. This allows us to assess visually whether the sample data plausibly came from the theoretical distribution.

Common Probability Distributions

To understand what Q-Q plots can tell us, it‘s helpful to review some common probability distributions and their characteristics. Here are a few distributions frequently encountered in data science:

Normal Distribution

The normal or Gaussian distribution is one of the most important probability distributions. It has a symmetric bell shape, with the mean, median and mode all equal and located at the center of the distribution. Many natural phenomena follow an approximately normal distribution.

The normal distribution is defined by two parameters:

• μ (mu) is the mean or expectation
• σ (sigma) is the standard deviation

The standard normal distribution is the normal distribution with a mean of 0 and standard deviation of 1, denoted as N(0,1).

Some key properties of normal distributions:

• Symmetric, with 50% of values below mean and 50% above
• Approximately 68% of values within 1 standard deviation of mean
• Approximately 95% of values within 2 standard deviations of mean
• Approximately 99.7% of values within 3 standard deviations of mean

Uniform Distribution

In a uniform distribution, all values over a specified range are equally likely. The probability density function is flat. Examples include the outcome of rolling a fair die (discrete) or the location of a randomly dropped object along a ruler (continuous).

The continuous uniform distribution has two parameters:

• a is the minimum value
• b is the maximum value

All values between a and b are equally likely, with probability density 1/(b-a). Values outside this range have probability 0.

Exponential Distribution

The exponential distribution models the time until an event occurs in a Poisson process, where events occur continuously and independently at a constant average rate. Examples include the time until a radioactive particle decays or a customer arrives at a store.

The exponential distribution has one parameter:

• λ (lambda) is the rate or inverse of the mean

The mean of an exponential distribution equals 1/λ. The distribution is skewed right, with a long tail.

Why Understanding Data Distributions Matters for Machine Learning

Many ML algorithms are parametric, meaning they assume a particular form for the distribution of the data. The performance and validity of these models can suffer if their assumptions are not met.

For example, linear regression models assume that:

• The relationship between features and target is linear
• The errors (residuals) are normally distributed with mean 0 and constant variance
• The errors are independent

Logistic regression assumes the error terms follow a logistic distribution. Naive Bayes classifiers assume the features are conditionally independent given the class.

If we fit models to data that violate their assumptions, we risk getting biased parameter estimates, unreliable predictions, and misleading inferences. Checking assumptions with Q-Q plots before selecting and fitting models can help avoid these problems.

Even non-parametric ML algorithms that don‘t make explicit distributional assumptions can still be affected by the shape of the data. For example, skewed distributions can impact model training by making it sensitive to outliers. Understanding your data‘s distribution can guide steps like feature scaling, outlier handling, and model evaluation.

Creating Q-Q Plots in Python

Now that we‘ve seen why Q-Q plots are useful, let‘s look at how to create them in Python. We‘ll use the statsmodels, NumPy, Matplotlib and Seaborn libraries.

First, let‘s generate some sample data from a normal distribution using NumPy:

import numpy as np

np.random.seed(1) #Set seed for reproducibility 
data = np.random.normal(loc=0, scale=1, size=1000)

This creates an array of 1000 random values drawn from a standard normal N(0,1) distribution.

To create the Q-Q plot, we‘ll use statsmodels:

import statsmodels.api as sm

sm.qqplot(data, line=‘45‘)

The line=‘45‘ argument draws a 45-degree reference line for comparison. Here the points fall very close to the reference line, indicating the sample data closely follows a normal distribution.

We can also compare the sample data to other theoretical distributions using the dist argument. Here‘s the same data compared to an exponential distribution with mean 1:

from scipy.stats import expon

sm.qqplot(data, dist=expon, line=‘45‘) 

The curvature of the points away from the reference line tells us the exponential distribution is not a good fit for this data.

Interpreting Q-Q Plots

When interpreting a Q-Q plot, focus on three key features:

1. Overall linearity: Do the points follow a roughly straight line? If so, the two distributions being compared are similar. Systematic curvature indicates distributional differences.

2. Tails: Look at the behavior of the points at the extremes (ends) of the plot. Divergence from the reference line in the tails can indicate the sample distribution has heavier or lighter tails than the theoretical distribution. Points above the line in the upper tail suggest a heavier right tail in the sample. Points below the line in the lower tail suggest a heavier left tail.

3. Slope: The slope of the Q-Q plot reflects the ratio of the sample and theoretical standard deviations. A slope > 1 means the sample standard deviation exceeds the theoretical. A slope < 1 means the theoretical standard deviation is greater.

Let‘s see some examples of how to interpret Q-Q plots for data with various distributions:

Normal Distribution

Here‘s a Q-Q plot of a sample drawn from a normal distribution with mean 50 and standard deviation 10, compared to a standard normal N(0,1) distribution:

data = np.random.normal(50, 10, size=1000)
sm.qqplot(data, line=‘45‘)

The points follow a straight line, but the slope is steeper than the 45-degree reference line. This tells us the sample data is normally distributed, but with a larger standard deviation than N(0,1). The intercept of the line is shifted right, reflecting the greater mean of the sample data.

Uniform Distribution

Here‘s 1000 values drawn from a uniform distribution over [0,1], plotted against a standard normal distribution:

data = np.random.uniform(0, 1, size=1000)
sm.qqplot(data, line=‘45‘)

The points follow an S-curve, indicating a distinctly non-normal distribution. The sample quantiles are less dispersed than expected for a normal distribution in both the lower and upper tails. This is a hallmark of uniform data, where all values are equally likely and there are hard boundaries on the range.

Exponential Distribution

Finally, let‘s compare a sample drawn from an exponential distribution with mean 2 to a standard normal:

data = np.random.exponential(scale=2, size=1000)  
sm.qqplot(data, line=‘s‘)

The curvature of the points indicates an exponential distribution. Notice how the sample quantiles increase rapidly relative to the theoretical normal quantiles in the upper tail. This reflects the long right tail of the exponential distribution.

Using Q-Q Plots to Validate Machine Learning Assumptions

We‘ve seen how Q-Q plots can characterize the distribution of a single variable. They are also an important tool for checking assumptions about model residuals in regression and other predictive models.

Recall that linear regression models assume the error terms are normally distributed with mean 0 and constant variance. If this assumption is violated, the regression coefficients and their standard errors may be biased or misleading.

Here‘s how to check this assumption with a Q-Q plot in Python. We‘ll use the built-in anscombe dataset, which shows the importance of visual data exploration. First, let‘s fit a linear regression model:

import statsmodels.formula.api as smf

data = sm.datasets.anscombe.load_pandas().data
model = smf.ols(‘y1 ~ x1‘, data=data).fit()

This fits a simple model predicting y1 from x1 in the first Anscombe quartet dataset.

To create a Q-Q plot of the residuals, access the model‘s resid attribute and pass it to sm.qqplot:

sm.qqplot(model.resid, line=‘45‘)

The residuals follow the reference line, indicating they are approximately normally distributed. There are no major outliers or curvature to suggest problems with the model assumptions.

Note that Q-Q plots don‘t tell the whole story. It‘s a good idea to complement them with other residual plots like residuals vs. fitted values to check for non-constant variance or non-linearity.

Conclusion

Q-Q plots are a powerful tool for visually comparing data distributions. They allow you to quickly assess whether sample data plausibly came from a particular theoretical distribution, and to compare the shapes of two datasets.

For machine learning practitioners, Q-Q plots have two key applications:

1. Exploring the distributions of features and target variables to guide model selection and data preparation.

2. Validating distributional assumptions of the models, such as normality of residuals.

By comparing theoretical and sample quantiles, Q-Q plots provide a quick way to characterize distributional shape, tail behavior, and adherence to assumptions. They can help ensure you are basing your models on the right probability distributions for your data.

The key steps to create and interpret Q-Q plots in Python are:

• Create sample data with numpy.random or load data into a DataFrame
• Call statsmodels.qqplot() on the data, specifying a theoretical distribution if desired
• Look for linearity to assess distributional similarity, curvature to identify distributional differences, and slope/intercept to compare parameters
• For regression models, check error term assumptions by creating a Q-Q plot of the .resid attribute against a normal distribution

Incorporating Q-Q plots into your data science workflow can help you better understand your data, select appropriate models, and ensure your model assumptions are met. While they are not a substitute for careful study design and data exploration, Q-Q plots are a handy tool to have in your diagnostic toolkit. By comparing distributions visually, they allow you to check model assumptions and select appropriate methods more easily.

I hope this post has helped explain the role of Q-Q plots in machine learning and how to create and interpret them in Python. The complete code for the examples is available on my Github. Happy modeling!