Using RFM and CLTV Analysis to Deeply Understand Your Customers

As a data-driven marketer or analyst, you know how crucial it is to deeply understand your customer base. Two of the most powerful techniques for customer analytics are RFM (Recency, Frequency, Monetary) analysis and CLTV (Customer Lifetime Value) modeling.

RFM segments customers based on how recently they made a purchase, how frequently they purchase, and how much they spend. It‘s a tried and true method for identifying your best customers and tailoring engagement accordingly.

CLTV goes a step further by predicting how much a customer will spend with your business over their entire relationship with you. Accurate CLTV predictions enable you to make smarter decisions about customer acquisition, retention, and targeting.

In this guide, I‘ll walk you through how to perform both RFM and CLTV analysis in Python. We‘ll work through a case study with an online retail dataset to illustrate the concepts. While we‘ll calculate RFM along the way, I‘ll focus more deeply on CLTV modeling, as it‘s an advanced technique with many nuances to understand.

By the end, you‘ll have the tools and code needed to apply RFM and CLTV to your own business. Let‘s dive in!

RFM Analysis: Segmenting Customers by Recency, Frequency and Monetary Value

RFM analysis ranks customers on three key dimensions:

Recency: How recently did the customer make a purchase? Customers who bought more recently are more likely to buy again.

Frequency: How frequently does the customer make purchases? Frequent buyers are more engaged and loyal.

Monetary: How much money has the customer spent in total? High spenders contribute more revenue.

To calculate RFM metrics:

1. Select a time period to analyze (often the past 1-2 years).
2. For each customer, calculate days since last purchase (Recency), number of orders in the time period (Frequency), and total spend (Monetary).
3. Rank all customers from 1-5 on each metric, with 5 being the highest value.
4. Combine the three scores (e.g. 5-1-4) to get a customer‘s overall RFM score.

Customers with similar RFM scores have similar purchase behavior, so we can group them into segments. Here‘s how to interpret some common "RFM personas":

Champions (5-5-5): Your best customers who bought recently, buy often, and spend a lot. Reward them and find ways to delight them.

At Risk (2-5-5): Big spenders who have lapsed recently. Get them back with targeted reactivation campaigns.

Potential Loyalists (5-3-3): Recent customers with average frequency and spend. Find ways to nurture them and increase their engagement.

It‘s straightforward to code up RFM in Python using the Pandas library. Here‘s a snippet to get you started:

import pandas as pd

# Load data 
data = pd.read_csv(‘customer_transactions.csv‘)
data[‘InvoiceDate‘] = pd.to_datetime(data[‘InvoiceDate‘])

# Calculate recency, frequency and monetary 
rfm = data.groupby(‘CustomerID‘).agg({
  ‘InvoiceDate‘: lambda x: (analysis_date - x.max()).days,
  ‘InvoiceNo‘: ‘count‘, 
  ‘TotalPrice‘: ‘sum‘})

rfm.rename(columns={
  ‘InvoiceDate‘: ‘Recency‘,
  ‘InvoiceNo‘: ‘Frequency‘, 
  ‘TotalPrice‘: ‘Monetary‘}, inplace=True)

# Generate RFM scores
quintiles = rfm[[‘Recency‘, ‘Frequency‘, ‘Monetary‘]].quantile([.2, .4, .6, .8]).to_dict()

def r_score(x):
    if x <= quintiles[‘Recency‘][.2]: return 5
    elif x <= quintiles[‘Recency‘][.4]: return 4
    elif x <= quintiles[‘Recency‘][.6]: return 3
    elif x <= quintiles[‘Recency‘][.8]: return 2
    else: return 1

def fm_score(x, c):
    if x <= quintiles[c][.2]: return 1
    elif x <= quintiles[c][.4]: return 2  
    elif x <= quintiles[c][.6]: return 3
    elif x <= quintiles[c][.8]: return 4
    else: return 5

rfm[‘R‘] = rfm[‘Recency‘].apply(lambda x: r_score(x))
rfm[‘F‘] = rfm[‘Frequency‘].apply(lambda x: fm_score(x, ‘Frequency‘))
rfm[‘M‘] = rfm[‘Monetary‘].apply(lambda x: fm_score(x, ‘Monetary‘))
rfm[‘RFM_Score‘] = rfm.R.map(str) + rfm.F.map(str) + rfm.M.map(str)

With RFM scores calculated, you can define customer segments and take appropriate marketing actions for each one. But while RFM is great for understanding past behavior, it doesn‘t explicitly predict future value. That‘s where CLTV comes in.

Predicting CLTV with Advanced Probability Models

The simplest way to calculate customer lifetime value is:

CLTV = Average Order Value × Purchase Frequency × Average Customer Lifespan

However, this assumes all customers are identical. Advanced CLTV models incorporate individual purchase patterns over time to make personalized predictions.

The best practice "buy ‘til you die" CLTV model has two components:

1. Estimate number of future transactions using the BG/NBD model. BG/NBD stands for the Beta Geometric / Negative Binomial Distribution model.
2. Estimate future average order value using the Gamma-Gamma model.

BG/NBD Model for Purchase Frequency

The BG/NBD model predicts the number of transactions a customer will make in the future based on their past transaction history. It does this by fitting two probability distributions to each customer‘s data:

1. The beta geometric distribution for purchase dropout probability. This models the chance a customer becomes inactive after each purchase.
2. The negative binomial distribution for purchase frequency. This models the number of purchases while active.

By combining these distributions, we get a prediction of the number of future purchases for each customer.

Gamma-Gamma Model for Monetary Value

The Gamma-Gamma model predicts a customer‘s average transaction value over their lifetime. Like the BG/NBD model, it fits two probability distributions:

1. The gamma distribution for the monetary value of individual purchases.
2. The gamma distribution for the average transaction value over the customer‘s lifetime.

Together, these distributions give us an estimated monetary value per transaction for each customer.

Putting It All Together

Once we have the estimated number of future transactions and average order value, we can multiply them together to get estimated CLTV:
CLTV = Predicted Number of Future Purchases × Predicted Average Purchase Value

Implementing BG/NBD and Gamma-Gamma models by hand would be extremely complex. Thankfully, the Lifetimes Python library makes it very straightforward:

from lifetimes import BetaGeoFitter, GammaGammaFitter

# Fit BG/NBD model
bgf = BetaGeoFitter(penalizer_coef=0.001)
bgf.fit(rfm[‘Frequency‘], rfm[‘Recency‘], rfm[‘T‘])

# Fit Gamma-Gamma model
returning_customers_summary = rfm[rfm[‘Frequency‘]>0]
ggf = GammaGammaFitter(penalizer_coef = 0.01)
ggf.fit(returning_customers_summary[‘Frequency‘],
        returning_customers_summary[‘Monetary‘])

# Predict 1-month and 12-month CLTV        
month_1_cltv = ggf.customer_lifetime_value(
    bgf, rfm[‘Frequency‘], rfm[‘Recency‘], rfm[‘T‘],
    rfm[‘Monetary‘], time=1, discount_rate=0.01)

month_12_cltv = ggf.customer_lifetime_value(
    bgf, rfm[‘Frequency‘], rfm[‘Recency‘], rfm[‘T‘],
    rfm[‘Monetary‘], time=12, discount_rate=0.01)   

We can add the CLTV predictions to our RFM table for a more complete view of customer value:

cltv_1_month = pd.DataFrame(month_1_cltv.sort_values(ascending=False)).reset_index()
cltv_1_month.columns = [‘CustomerID‘, ‘CLTV_1_Month‘]
cltv_data = rfm.merge(cltv_1_month, on=‘CustomerID‘, how=‘left‘)

cltv_12_month = pd.DataFrame(month_12_cltv.sort_values(ascending=False)).reset_index()
cltv_12_month.columns = [‘CustomerID‘, ‘CLTV_12_Month‘]
cltv_rfm_data = cltv_data.merge(cltv_12_month, on=‘CustomerID‘, how=‘left‘)

Combining RFM and CLTV for Customer Insights

With CLTV predictions added to our RFM table, we can analyze the alignment between past behavior and predicted future value.

Some questions to explore:

• Do our "Champion" customers based on past value also have the highest predicted CLTV?
• Which "At Risk" customers have high CLTV and may be worth trying to reactivate?
• How can we further segment customers based on the combination of RFM + CLTV?

As an example, let‘s see how our top 20% of customers based on past RFM compare to the top 20% based on predicted 12-month CLTV:

top_20_rfm = cltv_rfm_data[cltv_rfm_data[‘RFM_Score‘].isin([‘555‘,‘554‘,‘544‘,‘545‘,‘454‘])]
top_20_cltv = cltv_rfm_data.nlargest(int(len(cltv_rfm_data)*0.2), ‘CLTV_12_Month‘)

print("Percent of Top 20% RFM in Top 20% CLTV: ",
      len(top_20_rfm.merge(top_20_cltv, on=‘CustomerID‘)) / len(top_20_rfm))

In our example retail dataset, 73% of the top 20% RFM customers are also in the top 20% of predicted CLTV, showing fairly strong alignment between past and future value. However, over a quarter of top RFM customers don‘t have the highest future CLTV – so solely using RFM to target top customers may miss some high-potential accounts.

We can also create CLTV-based segments to complement RFM. Using 12-month predicted CLTV, let‘s split customers into four equal groups from highest to lowest value:

cltv_rfm_data[‘CLTV_Segment‘] = pd.qcut(cltv_rfm_data[‘CLTV_12_Month‘], 4, labels=[‘A‘,‘B‘,‘C‘,‘D‘])

Now we have a complete picture of each customer‘s historical behavior (RFM) and future value (CLTV). We can create highly targeted segments and more efficiently allocate marketing resources. For example:

• VIP segment: High RFM score (4-5) and top CLTV segment (A). These are your best customers to retain and expand.
• High potential segment: Moderate RFM score (3) and high CLTV segment (A-B). Find ways to move them into VIP status.
• Reactivation segment: Low recency score (1-2), but high CLTV segment (A-B). High-value customers at risk of churn who may be worth winning back.

Important Considerations for CLTV

While CLTV modeling is extremely valuable, it‘s important to keep a few things in mind:

1. Model performance: Before relying on CLTV predictions, validate model performance with metrics like mean squared error on a holdout set of data. The Lifetimes library includes functions for evaluating model fit.

2. Ongoing evaluation: Customer behavior evolves over time. Regularly update your models with new data to ensure predictions remain accurate.

3. Align with business reality: CLTV is a powerful input to decisions, but shouldn‘t be followed blindly. Consider factors the model doesn‘t account for, like product margins, retention costs, and brand impact.

4. Correlation vs. causation: A customer having high predicted CLTV doesn‘t necessarily mean marketing campaigns caused that high value. Use experiments to measure incremental impact.

Closing Thoughts

RFM and CLTV are two of the most effective techniques for understanding your customers. RFM helps you segment based on past behavior, while CLTV predicts future value. Combining both gives you a 360-degree view.

This guide walked through the why and how of RFM and CLTV:

• The components of RFM and how to segment customers with it
• The theory behind probabilistic CLTV models like BG/NBD and Gamma-Gamma
• How to implement CLTV modeling in Python with the Lifetimes library
• How to derive deeper insights from combined RFM + CLTV analysis

With this knowledge, you‘re well equipped to take your customer analytics to the next level. Start applying RFM and CLTV to your own customer data and see what valuable insights you uncover.

The better you understand your customers, the better you can serve them, and the more successful your business will be. Happy analyzing!