A Comprehensive Guide to Non-Stationary Time Series Analysis in Python
As an AI and machine learning expert, I‘ve seen firsthand how time series data is at the heart of many real-world applications, from forecasting stock prices and sales to anomaly detection in IoT sensor data. One of the first and most important steps in any time series analysis project is to check the stationarity of the data. Non-stationary time series pose unique challenges for modeling and require special handling. In this in-depth guide, we‘ll dive into the concept of stationarity, demonstrate techniques to detect non-stationarity, and walk through a complete case study of modeling non-stationary data in Python.
What is Stationarity?
A time series is said to be stationary if its statistical properties do not change over time. More formally, a time series {Xt} is stationary if it satisfies the following conditions:
- Constant mean: E(Xt) = μ for all t
- Constant variance: Var(Xt) = σ^2 for all t
- Constant autocovariance: Cov(Xt, Xt+h) = γ(h) for all t and h
In other words, the mean, variance, and autocovariance of a stationary series are time-invariant. They depend only on the lag h between observations, not on the absolute time t.
Here are some examples of stationary and non-stationary time series:
[Insert example plots of stationary and non-stationary series]Stationary series exhibit mean reversion and have a constant spread. Non-stationary series can have changing mean (trend), changing variance (heteroscedasticity), and seasonality.
Many real-world time series are non-stationary. Some common examples include:
- Stock prices: Stock prices often exhibit upward or downward trends and volatility clustering (periods of high and low variance).
- Sales data: Sales can have strong seasonality (e.g., spikes during holiday seasons) and trend due to business growth or decline.
- Temperature measurements: Daily temperature shows seasonality due to the cycle of seasons and long-term trend due to climate change.
Applying models that assume stationarity (like ARMA) to non-stationary data can give spurious results. The residuals will likely be correlated and the model will have poor out-of-sample performance. Therefore, it‘s crucial to check stationarity and transform the data if needed before fitting models.
Detecting Non-Stationarity
There are two main approaches to check if a time series is stationary: visual tools and statistical tests. Let‘s demonstrate them in Python.
Visual Inspection
The first step is to plot the time series and visually check for trends and seasonality. We can also examine the autocorrelation plot to see if the autocorrelation structure changes over time. Here‘s an example using the international airline passenger dataset:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from statsmodels.graphics.tsaplots import plot_acf
data = pd.read_csv(‘AirPassengers.csv‘)
data[‘Month‘] = pd.to_datetime(data[‘Month‘])
data.set_index(‘Month‘, inplace=True)
# Plot the time series
plt.figure(figsize=(12,6))
plt.plot(data)
plt.title(‘International Airline Passengers‘)
plt.xlabel(‘Year‘)
plt.ylabel(‘Passengers (1000s)‘)
plt.show()
# Plot the autocorrelation function
plot_acf(data, lags=36)
plt.show()The plot shows a clear upward trend and yearly seasonality, indicating non-stationarity. The ACF plot shows high autocorrelation even at long lags, another sign of non-stationarity.
Statistical Tests
While visual inspection is useful, statistical tests provide an objective measure of stationarity. The most widely used tests are the Augmented Dickey-Fuller (ADF) test and the Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test. Let‘s apply them to the airline passenger data using the statsmodels library:
from statsmodels.tsa.stattools import adfuller, kpss
# ADF test
result = adfuller(data[‘Passengers‘])
print(f‘ADF Statistic: {result[0]}‘)
print(f‘p-value: {result[1]}‘)
# KPSS test
result = kpss(data[‘Passengers‘])
print(f‘\nKPSS Statistic: {result[0]}‘)
print(f‘p-value: {result[1]}‘)ADF Statistic: 0.6500177430388159
p-value: 0.9887689367952038
KPSS Statistic: 2.1201435227215266
p-value: 0.01The ADF test has a null hypothesis of non-stationarity. A small p-value (e.g., less than 0.05) indicates strong evidence against the null, suggesting stationarity. Here, the large p-value of 0.989 fails to reject the null, indicating non-stationarity.
The KPSS test has a null hypothesis of stationarity. A small p-value suggests evidence against stationarity. The p-value of 0.01 leads us to reject the null, again concluding non-stationarity.
In summary, both visual inspection and statistical tests confirm that the airline passenger series is non-stationary. Next, we‘ll look at ways to transform it into a stationary series suitable for modeling.
Making a Time Series Stationary
There are three main methods to stationarize a time series:
- Differencing: Differencing can remove constant trends. We subtract each observation from the previous one.
- Seasonal Differencing: For data with stable seasonality, we can difference at the seasonal lag to remove seasonal effects.
- Transformations: Applying mathematical transformations like log, square root, or Box-Cox can stabilize the variance of a series.
Let‘s demonstrate each of these methods in Python.
Differencing
Here‘s how to difference the airline passenger data:
diff_data = data.diff().dropna()
plt.figure(figsize=(12,6))
plt.plot(diff_data)
plt.title(‘First Differenced Series‘)
plt.xlabel(‘Year‘)
plt.ylabel(‘Passengers (1000s)‘)
plt.show()Differencing has removed the constant upward trend, but the series still shows seasonality. The ADF test now rejects the null of non-stationarity (p-value < 0.05), but the KPSS test still indicates non-stationarity due to seasonality.
result = adfuller(diff_data[‘Passengers‘])
print(f‘ADF Statistic: {result[0]}‘)
print(f‘p-value: {result[1]}‘)ADF Statistic: -4.239109375716928
p-value: 0.0006876139881405119Seasonal Differencing
To remove yearly seasonality, we can seasonally difference by taking each observation minus the observation from the same month in the previous year:
diff_seasonal_data = data.diff(periods=12).dropna()
plt.figure(figsize=(12,6))
plt.plot(diff_seasonal_data)
plt.title(‘Seasonally Differenced Series‘)
plt.xlabel(‘Year‘)
plt.ylabel(‘Passengers (1000s)‘)
plt.show()The seasonally differenced series looks more stationary. Both the ADF and KPSS tests now conclude stationarity.
Transformations
Differencing removes trends but doesn‘t stabilize variance. A common issue is heteroscedasticity, where the variance of the series changes over time.
The airline passenger data exhibits increasing variance, with larger fluctuations later in the series. We can stabilize the variance with a log transformation:
log_data = np.log(data)
plt.figure(figsize=(12,6))
plt.plot(log_data)
plt.title(‘Log Transformed Series‘)
plt.xlabel(‘Year‘)
plt.ylabel(‘Log Passengers‘)
plt.show()
# seasonal differencing of log data
diff_seasonal_log_data = log_data.diff(periods=12).dropna()
plt.figure(figsize=(12,6))
plt.plot(diff_seasonal_log_data)
plt.title(‘Seasonally Differenced Log Series‘)
plt.xlabel(‘Year‘)
plt.ylabel(‘Log Passengers‘)
plt.show()The log transformed data has more stable variance over time. Seasonally differencing the log data yields a series that looks stationary and passes the ADF and KPSS tests.
For more complex patterns of non-constant variance, a Box-Cox transformation can help:
from scipy.stats import boxcox
data_boxcox, _ = boxcox(data[‘Passengers‘])
plt.figure(figsize=(12,6))
plt.plot(data_boxcox)
plt.title(‘Box-Cox Transformed Series‘)
plt.xlabel(‘Year‘)
plt.ylabel(‘Box-Cox Passengers‘)
plt.show() The Box-Cox transformation further equalizes the variance across the series.
Case Study: Forecasting with ARIMA
Now that we‘ve stationarized the airline passenger data, let‘s fit an ARIMA model and forecast future values. ARIMA is a powerful model for univariate time series that combines autoregression (AR), differencing (I), and moving average (MA) components.
We‘ll use the statsmodels library to fit an ARIMA(1,1,1)(1,1,1)[12] model, which includes:
- 1 nonseasonal AR lag
- 1 nonseasonal differencing
- 1 nonseasonal MA lag
- 1 seasonal AR lag
- 1 seasonal differencing
- 1 seasonal MA lag
- Seasonality of 12 (monthly data)
Here‘s the code to fit the model and generate forecasts:
from statsmodels.tsa.arima_model import ARIMA
# Fit the model
model = ARIMA(data, order=(1,1,1), seasonal_order=(1,1,1,12))
results = model.fit()
print(results.summary())
# Generate forecasts
start_date = data.index[-1] + pd.DateOffset(months=1)
end_date = data.index[-1] + pd.DateOffset(months=24)
forecast = results.predict(start=start_date, end=end_date)
# Plot the forecasts
plt.figure(figsize=(12,6))
plt.plot(data, label=‘Observed‘)
plt.plot(forecast, label=‘Forecast‘)
plt.title(‘ARIMA Forecast‘)
plt.xlabel(‘Year‘)
plt.ylabel(‘Passengers (1000s)‘)
plt.legend()
plt.show()The model diagnostics look good, with uncorrelated residuals and significant coefficients. The forecasts capture the upward trend and seasonal pattern.
We could further tune the model by testing different lag orders or including exogenous variables, but this illustrates the general process of fitting an ARIMA to non-stationary data.
Advanced Topics and Further Reading
This guide covered the essentials of non-stationary time series, but there are many more advanced techniques to explore:
- SARIMA (Seasonal ARIMA) for data with complex seasonality
- Vector Autoregression (VAR) for multivariate non-stationary series
- Unit root tests with structural breaks
- Cointegration for non-stationary series that share a common stochastic trend
- Nonlinear models like ARCH and GARCH for heteroscedastic data
Here are some resources to dive deeper:
- Unit Roots, Cointegration, and Structural Change by G. S. Maddala
- Non-Stationary Time Series Analysis and Cointegration by Terence C. Mills
- tsibble and feasts R packages for tidy time series analysis
- Applied Statistical Time Series Analysis course by Robert Nau
Conclusion
In this guide, we took a deep dive into non-stationary time series analysis in Python. We covered:
- The definition and importance of stationarity
- Visual and statistical methods to test for stationarity
- Techniques like differencing and transformations to make series stationary
- A case study of fitting an ARIMA model to non-stationary data
Stationarity is a key concept in time series analysis. Non-stationary data can lead to spurious regressions and inaccurate forecasts. By carefully checking for stationarity and applying the appropriate transformations, we can build reliable models that capture the underlying dynamics of the data.
Time series analysis is a vast and complex field, but I hope this guide provided a solid foundation to help you approach non-stationary data with confidence. The skills covered here are widely applicable, from forecasting economic indicators to anomaly detection in streaming data.
As you continue your journey into time series modeling, remember that stationarity is just one piece of the puzzle. The most effective models combine domain expertise, careful feature engineering, and rigorous model validation. But with a strong understanding of stationarity in your toolkit, you‘ll be well-equipped to tackle even the messiest real-world time series.
