A Deep Dive into Statistical Tests for Stationarity in Time Series: The Augmented Dickey-Fuller Test

Introduction

When working with time series data, one of the first and most important things to assess is whether the series is stationary. Stationarity is a property of a time series where the statistical properties such as the mean, variance, and autocorrelation structure do not change over time. In other words, the series does not exhibit trends or seasonal patterns and does not have a changing variance.

Stationarity is a crucial assumption for many statistical models and forecasting methods applied to time series. Models like ARMA and ARIMA are designed for stationary series, while methods such as exponential smoothing handle certain types of non-stationarity. Applying a model that assumes stationarity to a non-stationary series can lead to spurious results. Therefore, it‘s essential to check for stationarity before selecting an appropriate modeling approach.

While stationarity is clearly important, it can sometimes be difficult to determine whether a given time series possesses this property. The good news is that there are reliable statistical tests that allow us to formally assess stationarity. In this post, we‘ll take an in-depth look at one of the most widely used tests: the augmented Dickey-Fuller (ADF) test.

Understanding Stationarity

Before we dive into the details of stationarity tests, let‘s make sure we have a solid understanding of what stationarity means. There are two types of stationarity that are often discussed: strict (or strong) stationarity and weak (or covariance) stationarity.

A time series is said to be strictly stationary if the joint probability distribution of any set of observations does not depend on time. This is a very strong condition that is difficult to verify empirically. In practice, we typically focus on weak stationarity, which is a less restrictive assumption.

A weakly stationary time series has the following properties:

1. The mean of the series is constant over time
2. The variance of the series is constant over time (homoscedasticity)
3. The covariance between any two observations depends only on the time lag between them, not on the actual times

In essence, a stationary series will not exhibit trends, seasonality, or changes in variance over time. The autocorrelation function (ACF) and partial autocorrelation function (PACF) of a stationary series will also decay quickly to zero.

Visual Diagnostics for Stationarity

One of the first steps in assessing stationarity should be to visually examine plots of the time series. There are several telltale signs that a series is non-stationary:

• An obvious upward or downward trend over time
• Clear seasonal patterns that repeat at fixed intervals
• Increasing or decreasing variance as time progresses

For example, consider the following plots:

[Insert plots showing trended, seasonal, and heteroscedastic time series]

The first plot shows a time series with a clear upward trend, indicating that the mean is not constant over time. The second plot exhibits strong seasonality, with a repeating pattern that occurs at a fixed frequency. The third plot demonstrates changing variance, with the fluctuations becoming more pronounced as time goes on. All three of these series would be considered non-stationary.

While visual inspection can provide clues about stationarity, it is not always definitive. Time series can be noisy, making it difficult to discern patterns with the naked eye. Some non-stationary series may appear stationary over short time periods. To complicate matters further, a time series may be stationary in some ways but not others. For example, a series could have a constant mean but non-constant variance.

To rigorously determine whether a time series is stationary, we need to rely on statistical tests. One of the most popular and powerful options is the augmented Dickey-Fuller test.

The Augmented Dickey-Fuller Test

The augmented Dickey-Fuller (ADF) test is a type of unit root test that is widely used to assess the stationarity of a time series. The basic idea behind unit root tests is that non-stationary series are characterized by the presence of a stochastic trend or unit root.

The ADF test is an extension of the original Dickey-Fuller test, which was developed by statisticians David Dickey and Wayne Fuller in 1979. The original test was based on a simple autoregressive (AR) model:

y_t = ρy_t-1 + ε_t

where y_t is the value of the series at time t, ρ is the autoregressive coefficient, and ε_t is white noise.

The null hypothesis of the Dickey-Fuller test is that ρ = 1, indicating the presence of a unit root and thus non-stationarity. The alternative hypothesis is that |ρ| < 1, meaning the series is stationary.

The augmented Dickey-Fuller test expands on this basic framework by adding lagged difference terms to the model:

Δy_t = α + βt + γy_t-1 + δ₁Δy_t-1 + … + δ_p-1Δy_t-p+1 + ε_t

Here, Δ denotes the first difference operator, so Δy_t = y_t – y_t-1. The terms Δy_t-1, …, Δy_t-p+1 are the p-1 lagged differences, which are included to account for potential serial correlation in the error terms.

The ADF test statistic is calculated as the t-statistic for the coefficient γ. If γ is significantly less than zero, we can reject the null hypothesis of a unit root and conclude that the series is stationary.

Implementing the ADF Test in Python

Carrying out the ADF test in Python is straightforward thanks to the statsmodels library. Here‘s a simple example using the annual sunspot data:

from statsmodels.tsa.stattools import adfuller
import numpy as np
import pandas as pd

# Load the sunspot data
data = sm.datasets.sunspots.load_pandas().data
data.index = pd.Index(sm.tsa.datetools.dates_from_range(‘1700‘, ‘2008‘))

# Perform the ADF test
result = adfuller(data[‘SUNACTIVITY‘])

print(f‘ADF Statistic: {result[0]}‘)  
print(f‘p-value: {result[1]}‘)
print(‘Critical Values:‘)  
for key, value in result[4].items():
    print(f‘{key}: {value}‘)  

This code first loads the sunspot data and converts the index to a datetime format. It then applies the adfuller function to the SUNACTIVITY column, which contains the annual sunspot counts.

The adfuller function returns a tuple with several output values. The first is the test statistic, and the second is the p-value. The fourth value is a dictionary of critical values for different significance levels.

Here‘s the output from running this code:

ADF Statistic: 2.8377807737032556
p-value: 0.053075887244556114
Critical Values:  
1%: -3.4523369032869326
5%: -2.8712225944087346
10%: -2.5719291964635066

To determine whether we can reject the null hypothesis of non-stationarity, we compare the test statistic to the critical values and examine the p-value. Here, the test statistic is greater than the critical values at all significance levels. The p-value is also larger than common significance thresholds like 0.01 or 0.05.

Therefore, in this case, we fail to reject the null hypothesis. The evidence suggests that the sunspot series is non-stationary.

The KPSS Test

While the ADF test is the most commonly used method for assessing stationarity, it‘s often a good idea to employ multiple tests for robustness. One popular alternative is the Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test.

The key difference between the ADF and KPSS tests lies in their null and alternative hypotheses. While the ADF test has a null hypothesis of non-stationarity, the KPSS test assumes stationarity under the null. More specifically:

• ADF test:
  • H₀: series is non-stationary / has a unit root
  • H₁: series is stationary
• KPSS test:
  • H₀: series is trend-stationary
  • H₁: series is non-stationary

Due to their opposing null hypotheses, the ADF and KPSS tests can sometimes lead to conflicting results. There are a few potential explanations for this:

1. The series may be trend-stationary. In this case, the ADF test would correctly reject the null hypothesis of a unit root, but the KPSS test may fail to reject the null of trend stationarity. Removing the trend through detrending techniques like regression can often make the series strictly stationary.

2. The series may be difference-stationary. Here, the KPSS test would correctly reject the null of trend stationarity, but the ADF test may fail to reject the unit root null. Differencing the series one or more times can eliminate the non-stationarity.

3. The series may be fractionally integrated, exhibiting long memory properties. Such series can appear stationary or non-stationary depending on the test used.

Given these complexities, it‘s generally recommended to apply both the ADF and KPSS tests when assessing stationarity. If the results agree, we can be more confident in our conclusions. If they conflict, we may need to investigate further to determine the specific nature of the non-stationarity.

Conclusion

Checking for stationarity is a crucial first step in any time series analysis. While visual methods can provide some intuition, statistical tests like the augmented Dickey-Fuller (ADF) test offer a rigorous way to assess whether a series contains a unit root and is therefore non-stationary.

In this post, we took a deep dive into the ADF test, explaining the intuition behind the test, how it works, and how to implement it in Python. We also discussed the KPSS test as a complementary method and provided guidance on interpreting conflicting test results.

It‘s important to keep in mind that stationarity tests are not always definitive. Time series can exhibit complex behavior that may not be fully captured by these tests. However, they remain essential tools in the time series analyst‘s toolkit.

If a series is found to be non-stationary, there are various techniques for inducing stationarity, such as detrending, differencing, and seasonal adjustment. We‘ll explore these methods in more detail in future posts.

By carefully assessing stationarity and applying appropriate transformations when needed, we can ensure that our time series models are valid and produce reliable forecasts. The augmented Dickey-Fuller test is a key ally in this endeavor.