Feature Engineering Techniques For Time Series Data

6 Powerful Feature Engineering Techniques for Time Series Forecasting with Python

Introduction
Time series forecasting is an important problem that comes up in many different fields, from retail demand planning to weather prediction to stock market analysis. While classic time series models like ARIMA can be effective, they often underperform machine learning models that are able to capitalize on a richer set of input features beyond just the target time series itself.

This is where feature engineering comes in. By constructing new variables that capture important patterns and relationships in the data, we can dramatically improve the predictive power of our models. Feature engineering is part art, part science – it requires a mix of domain knowledge, intuition, and iteration to find the right transformations.

In this article, we‘ll walk through 6 essential techniques for time series feature engineering with detailed code examples in Python. Whether you‘re a data scientist getting started with time series or a seasoned practitioner looking to up your game, there will be something here for you. Let‘s dive in!

Understanding Time Series Data
Before we jump into specific techniques, it‘s important to understand some key characteristics that distinguish time series data:

Trend: Does the data tend to increase or decrease over time? Many time series exhibit long-term trends.

Seasonality: Are there regular, repeating patterns? For example, retail sales tend to peak around the holiday season each year.

Cycles: Are there longer-term, multi-year patterns like economic boom-and-bust cycles?

Nonstationarity: Is the distribution of the data changing over time? Many real-world time series have properties like mean and variance that are not constant.

Autocorrelation: Are observations correlated with previous values in the series? Most time series exhibit some degree of autocorrelation.

Keeping these common patterns in mind will help guide our feature engineering approach. Many techniques are designed to explicitly capture trending, seasonal or autocorrelated behavior.

1. Date and Time Features

The most basic type of feature engineering for time series data is to extract additional information from the timestamp itself. While a datetime is usually represented as a single entity, it actually contains multiple pieces of information:

• Calendar date
• Time of day
• Day of week
• Month
• Year
• Holiday

For example, let‘s say we‘re working with hourly retail sales data. Knowing whether a data point lands on a weekday vs weekend or in December vs July is highly relevant! We can easily parse datetime stamps to extract these attributes:

import pandas as pd

def extract_date_features(df, date_col):
df[date_col] = pd.to_datetime(df[date_col])
df[‘month‘] = df[date_col].dt.month
df[‘day_of_month‘] = df[date_col].dt.day
df[‘day_of_week‘] = df[date_col].dt.dayofweek
df[‘hour‘] = df[date_col].dt.hour
df[‘is_weekend‘] = (df[‘day_of_week‘] >= 5).astype(int)

return df

Adding date and time features is a simple transformation, but can be quite powerful, especially when dealing with data that has human-scale seasonality. In addition to explicitly extracting date parts, another useful technique is to encode cyclical features like hour of day or month of year using sine and cosine transforms – this allows tree-based models to more easily learn seasonal patterns.

2. Lag Features

Lag features are one of the most commonly used types of time series features, especially for autoregressive models. The idea is simple: use previous values of the target variable as features for the current time step. For example, we can create a lag-1 feature using Pandas like this:

df[‘lag_1‘] = df[‘sales‘].shift(1)

By including lag features, we‘re essentially giving the model access to the recent history of the time series, which can help it identify patterns and anticipate future values. You can create multiple lag features at different scales (hourly/daily/weekly lags for example) to capture both short and long-range dependencies.

The downside of lag features is that they obviously can‘t be generated for the first few time steps where prior values are not available. This means we have to drop some initial rows of data – not a huge issue with long time series but problematic if data is limited. They also tend to be highly correlated with each other, so using too many lags can lead to overfitting and multicollinearity issues.

In practice, the number of lags to use depends on the problem – you can select them via cross-validation or by analyzing autocorrelation and partial autocorrelation plots to identify significant time scales.

3. Rolling Window Statistics

Rolling window operations are an expansion of lag variables that calculate a statistic across multiple previous time steps. Common examples include rolling means, rolling standard deviations, and rolling min/max.

Here‘s how to calculate a 7-day rolling average in Pandas:

df[‘sales_rolling_7d_mean‘] = df[‘sales‘].rolling(window=7).mean()

Why are rolling windows useful? They act as a simple way to smooth out noise and detect trends. Rolling averages in particular are used in many classic decomposition methods like STL to separate trend/seasonality from residuals.

You can get creative with window functions too – rolling quantiles, entropy, kurtosis, etc. can all generate interesting features. Just be cautious of leaking future information when using rolling windows – the window at time t should only include data from timepoints before t.

4. Exponentially Weighted Moving Averages

Exponentially weighted moving averages (EMAs) are a variation on rolling means that assign greater weight to more recent observations. Whereas a simple rolling mean weights all points equally, EMAs apply an exponentially decaying weight to earlier timepoints.

There are a few reasons you might prefer EMAs to rolling averages:

• They‘re more responsive to recent changes in the data while still smoothing out noise
• They can be calculated recursively which is computationally efficient
• The weighting captures the intuition that recent points matter more for future predictions
  Here‘s how to add EMA features in Pandas:
  from statsmodels.tsa.api import ExponentialMovingAverage

def add_ema(df, col, span):
ema = ExponentialMovingAverage(df[col], span=span)
df[f‘{col}ema{span}‘] = ema.mean()

return df

A span of 5 corresponds to a half-life of about 3.5 timesteps, meaning points 3-4 steps back in time will have 50% as much influence as the most recent value. In general, shorter spans lead to more reactive, noise-sensitive EMAs while longer spans produce more stable, slow-moving averages. You can optimize the span length via backtesting.

5. Interaction Features

Interaction features try to capture relationships between different input variables. In a time series context, we often care about interactions between timestamp information and other features.

For example, suppose we‘re predicting demand for Uber rides. There are clear main effects related to the time of day (rush hour is busier) and the weather (rain drives up demand). But there are also likely interaction effects between these two variables – e.g. the peak during rainy rush hours will be higher than the sum of the individual time of day and weather effects.

We can create interaction features by simply multiplying variables together:

df[‘rush_hour_X_rain‘] = df[‘is_rush_hour‘] * df[‘is_raining‘]

You can get more sophisticated by defining custom functions that combine multiple variables. The key is to think about which combinations of features might have interesting relationships.

One word of caution: interaction terms tend to be correlated with their component features, so watch out for issues like multicollinearity. Regularization techniques can help with this.

6. Domain-Specific Features

Perhaps the most impactful type of time series feature engineering is creating custom, domain-specific indicators based on your knowledge of the problem. While the previous techniques can be applied to any time series, the features you derive from careful analysis of a specific domain will often be the real game-changers.

For example:

• If forecasting sales, create features around major holidays, new product launches, and competitor promotions
• For predicting hospital patient volume, look at local demographic trends, flu season indicators, and scheduled events like marathons
• With economic time series, add features related to government spending, consumer sentiment indexes, and treasury rates

The options are endless and depend entirely on the nuances of your particular dataset and problem. This is where you can get creative and rely on your domain expertise to brainstorm ideas.

That said, there are a few general principles to keep in mind:

Intuition is great, but also use data to validate your ideas. Don‘t be afraid to create wacky features, but check whether they actually improve performance before including them in a model.

Start simple and iterate. Begin with basic features and progressively add complexity, rather than throwing everything but the kitchen sink in at the start.

Consult the academic literature and industry benchmarks. Chances are someone has tackled a similar problem before – use their work as inspiration.

Wrapping Up
We‘ve covered a lot of ground in this article, but hopefully you now have a solid framework for approaching feature engineering on your next time series project. As a final tip – make sure to always use a separate validation set when testing out feature engineering ideas to avoid overfitting.

Time series forecasting is a complex challenge, but with the right approach to feature creation, you can uncover powerful signals to drive model performance. Don‘t be afraid to get creative, experiment, and iterate. And above all, let the data be your guide.

Thanks for reading! Feel free to connect with any questions or ideas.