Multi-variate Time Series Forecasting with the Kats Model: A Comprehensive Guide

Time series forecasting is a crucial area of machine learning that involves predicting future values based on historical time-based data. It has many important business applications, from forecasting sales and product demand to predicting equipment failures, stock prices, weather and more. While many time series problems involve forecasting a single variable, in reality most scenarios require predicting multiple, related variables – this is known as multi-variate time series forecasting.

In this guide, we‘ll take an in-depth look at multi-variate time series forecasting using the Kats model, an open-source Python library developed by Facebook. We‘ll cover the key concepts, walk through a detailed example, and share tips and best practices to achieve the best results. Whether you‘re a data scientist, analyst, or business leader, understanding how to apply multi-variate time series forecasting can unlock valuable insights to drive smarter decisions. Let‘s dive in!

What is Multi-variate Time Series Forecasting?

A time series is a sequence of data points recorded over consistent intervals of time. With time series forecasting, the goal is to predict future data points based on the patterns and trends in the historical data. Most introductory examples focus on univariate time series analysis – i.e. forecasting a single variable like sales or stocks.

However, in the real-world, multiple factors can impact the variable we want to predict. For instance, sales of a product may depend on its pricing, marketing spend, seasonality, and competitor activity. To make an accurate forecast, we need to consider the relationships between these variables. This is multi-variate time series forecasting.

With a multi-variate forecast, we use multiple input variables to predict one or more output variables. The predictors can influence each other as well as the target variable. This is important for capturing more complex patterns compared to univariate analysis.

Some common use cases of multi-variate time series forecasting include:

• Demand forecasting based on price, promotions, economic factors etc.
• Forecasting equipment maintenance needs based on usage, environmental conditions, and operating metrics
• Predicting flight delays based on weather, airport congestion and other factors
• Estimating future store sales based on product assortment, pricing, and local economic indicators

As you can see, multi-variate time series forecasting has broad applicability across industries and domains. Virtually any scenario where multiple time-based variables interact with each other could be a candidate.

Introducing the Kats Model for Time Series Forecasting

Kats is an open-source Python library developed by Facebook for time series analysis and forecasting. It provides a set of tools and models designed to simplify the end-to-end process of working with time series data.

Some key features of Kats include:

• Models for forecasting, detecting trends, seasonality and anomalies
• Utilities for time series manipulation, resampling, and featurization
• Evaluation and visualization functions for model analysis
• Support for both univariate and multi-variate time series forecasting

Kats follows a modular architecture, so you can mix and match components for your specific use case. It also integrates with other popular libraries like Pandas and scikit-learn.

For multi-variate time series forecasting, Kats provides an implementation of vector autoregression (VAR). Before we look at how to use it, let‘s briefly review what VAR is and how it works.

Understanding Vector Autoregression (VAR)

Vector autoregression is a statistical model used for multi-variate time series analysis. It captures the linear relationships between multiple variables over time.

The key idea behind VAR is that each variable depends on:

1. Its own lagged (past) values
2. The lagged values of the other variables

Mathematically, a VAR model with p lags (VAR(p)) can be written as:

y(t) = c + A1y(t-1) + A2y(t-2) + … + Ap*y(t-p) + e(t)

where:

• y(t) is a vector representing the values of the variables at time t
• c is a vector of constants
• Ai is a matrix of coefficients for the i-th lag
• e(t) is a vector of error terms

Intuitively, this equation says that the value of each variable at time t is a linear combination of the lagged values of itself and the other variables, plus some error term.

The coefficients in the Ai matrices capture the influence of each lagged variable on the others. This allows VAR to model the bi-directional relationships between the variables. In other words, the variables can affect each other, not just the target variable.

To use VAR for forecasting, we first estimate the coefficients using the historical data. Then we can recursively make predictions by plugging in lagged values from previous time steps.

Compared to univariate models like ARIMA, VAR can capture more complex, multivariate patterns. However, it does assume the relationships between variables are linear and stationary over time.

Now let‘s see how to implement a VAR model using Kats and apply it to a practical example.

Hands-on Example: Forecasting Sales and Visits with Kats

To illustrate how to use Kats for multi-variate time series forecasting, we‘ll walk through an example of predicting monthly sales and visits for a product based on historical data. We‘ll follow these steps:

1. Prepare the data
2. Create and train the model
3. Make predictions
4. Evaluate the model‘s performance

Step 1: Preparing the Data

First we need to load and structure our input data properly. Kats expects the data in a Pandas DataFrame with a DatetimeIndex and the variables as columns.

import pandas as pd
from kats.consts import TimeSeriesData

data = pd.read_csv(‘product_sales.csv‘)
data[‘Date‘] = pd.to_datetime(data[‘Date‘])
data = data.set_index(‘Date‘)

ts_data = TimeSeriesData(data)

Here we load the raw data from a CSV, convert the date column to datetime format, set it as the index, and wrap it in a Kats TimeSeriesData object.

Step 2: Creating and Training the Model

Next we create an instance of the VAR model and train it on our data. We need to specify the number of lags (p) to use – this is a hyperparameter we can tune.

from kats.models.var import VARModel, VARParams

params = VARParams(p=3)
model = VARModel(data=ts_data, params=params)
model.fit()

Here we use 3 lags and fit the model to estimate the coefficients.

Step 3: Making Predictions

To make forecasts, we simply call the model‘s predict method with the desired number of steps ahead to predict:

steps = 6
fcst = model.predict(steps=steps, include_history=True)
fcst = fcst.pd_dataframe()
fcst

This returns a DataFrame with the forecasted values for each variable.

Step 4: Evaluating Performance

Finally, we can assess how well the model performed by comparing the predictions to the actual values on a held-out test set. Some common evaluation metrics for multi-variate forecasts include:

• Mean Absolute Percentage Error (MAPE)
• Mean Absolute Error (MAE)
• Mean Squared Error (MSE)
• Weighted Absolute Percentage Error (WAPE)

Here‘s how we can calculate MAPE:

from kats.metrics.metrics import mean_absolute_percentage_error

actuals = test_data.to_dataframe().reset_index(drop=True)
mape = mean_absolute_percentage_error(actuals, fcst)
print(f‘MAPE: {mape:.2f}‘)

We can also plot the forecasts against the actual values to visually assess the fit:

import matplotlib.pyplot as plt

ax = fcst.plot(xlabel=‘Date‘, ylabel=‘Visits‘, figsize=(10, 6))
actuals.plot(ax=ax, style=‘k.‘, label=‘Actual‘) 
plt.legend()
plt.show()

This plots the historical data, forecasts, and actual values together.

Tips and Best Practices

Here are some tips and best practices to keep in mind when using Kats for multi-variate time series forecasting:

1. Ensure your input data is properly formatted and cleaned. Kats expects regularly spaced time intervals without missing values.

2. Normalize or standardize your variables if they have very different scales. This can help the model converge more easily.

3. Pay attention to the order of variables – the first one listed is treated as the primary one that influences the others.

4. Experiment with different numbers of lags and choose the value that minimizes your loss function on a validation set.

5. Be cautious when interpreting the model coefficients. While they can provide insight into variable relationships, correlation does not imply causation.

6. Make sure to evaluate your model‘s performance on an unseen test set that was not used during training, and compare it to a baseline.

7. While VAR can capture complex relationships, it assumes they are linear and stationary. If your data has non-linear or time-varying patterns, you may need a more sophisticated model.

8. Consider using techniques like rolling-window forecasting to retrain your model over time and adapt to new patterns.

Following these guidelines can help you get the most out of your multi-variate time series model.

Other Methods and Libraries

While Kats is a powerful and easy-to-use library for multi-variate time series forecasting, there are other techniques and tools to be aware of:

• Long Short-Term Memory (LSTM) networks are a type of recurrent neural network well-suited to sequences and time series. They can automatically learn non-linear patterns.

• Facebook Prophet is another open-source library for time series forecasting developed by Facebook. It uses a decomposable additive model and can incorporate regressors, but it is geared more towards univariate series.

• Classical methods like ARIMA and Holt-Winters exponential smoothing can be extended to multi-variate series as well.

• For more general-purpose machine learning, scikit-learn provides functions for multi-output regression that can be applied to time series.

The best approach will depend on your specific data, computational constraints, and accuracy requirements. It‘s often valuable to experiment with multiple methods and compare their performance.

Real-world Applications

We‘ve walked through a product sales example in this guide, but multi-variate time series forecasting has diverse applications across industries. Some examples include:

• Retail and CPG companies forecasting demand based on product, price, promotion, and demographic factors

• Banks predicting loan defaults and financial risks based on economic indicators, customer data, and market conditions

• Manufacturers forecasting equipment maintenance needs and failures based on sensor data, usage metrics, and environmental factors

• Transportation companies predicting transit demand based on route, weather, special events, and economic activity

• Healthcare providers forecasting patient volume based on demographics, seasonality, and local health trends

The ability to incorporate multiple variables, detect patterns, and anticipate future behavior adds a valuable dimension to planning and optimization.

Conclusion and Next Steps

In this guide, we‘ve taken a deep dive into multi-variate time series forecasting using the Kats library. We covered the key concepts, the vector autoregression (VAR) model, and walked through a step-by-step product demand forecasting example.

Some key takeaways include:

• Multi-variate time series forecasting predicts multiple, related variables and captures their interactions over time
• Vector autoregression (VAR) models each variable as a linear function of the lagged values of itself and the other variables
• The Kats library simplifies multi-variate time series forecasting in Python and integrates with Pandas
• Properly preparing data, choosing lags, and evaluating results on a test set are crucial for success
• Multi-variate forecasting has valuable applications across industries for demand, maintenance, risk prediction and more

If you‘re working on a forecasting problem with multiple variables, the Kats VAR model is a great place to start. However, it‘s also important to be aware of other methods like LSTM and experiment with different approaches.

To learn more and continue your multi-variate time series forecasting journey, here are some helpful resources:

• The official Kats documentation and tutorials: https://facebookresearch.github.io/Kats/
• "Forecasting: Principles and Practice" online textbook: https://otexts.com/fpp3/
• "Practical Time Series Analysis" book by Aileen Nielsen: https://www.oreilly.com/library/view/practical-time-series/9781492041641/

Time series forecasting is a complex but rewarding field. Multi-variate methods like VAR expand the range of problems we can tackle. With the right tools and approaches, you can uncover valuable insights to optimize decisions, streamline operations and delight customers. Happy forecasting!