Exploratory Data Analysis for Lead Scoring: Driving Revenue Growth with Data Science

Introduction

In today‘s hyper-competitive B2B landscape, efficiently identifying and prioritizing sales leads is table stakes for driving growth. Companies that excel at lead qualification enjoy higher conversion rates, larger deal sizes, and ultimately more revenue. A study by MarketingSherpa found that implementing lead scoring increased Microsoft‘s lead qualification rate by 300%.

Lead scoring is the process of assigning values to leads based on their likelihood to become customers. Scores are typically based on a combination of demographic attributes (job title, company size, industry, etc.) and behavioral data (website visits, email opens, event attendance). Marketers define the scoring rules and thresholds, often with input from the sales team.

Exploratory Data Analysis (EDA) is a critical first step in building an effective lead scoring model. EDA is the process of diving deep into the data to uncover insights, relationships, and patterns that can guide feature selection and model development. A thorough EDA lays the foundation for a highly predictive model.

In this post, we‘ll walk through a comprehensive EDA on a lead scoring dataset. We‘ll discuss data preparation, univariate and multivariate analysis, key findings, and recommendations. By the end, you‘ll have a framework for conducting your own lead scoring EDA and extracting actionable insights. Let‘s dive in!

The Dataset

Our dataset contains lead records collected by a mid-size B2B SaaS company over the past year. The company provides project management software and primarily targets small to medium businesses. The data contains a mix of demographic and engagement features on 50,000 leads, along with whether each lead converted to an opportunity. Here‘s a snapshot of the raw data:

Company Size	Industry	Job Title	Country	Website Visits	Webinar Attendee	Whitepaper Download	Trial Signup	Converted
Small	Technology	Manager	US	12	Yes	No	Yes	Yes
Medium	Healthcare	Director	UK	5	No	Yes	No	No
Large	Finance	VP	US	20	Yes	Yes	Yes	Yes
Small	Retail	Manager	CA	2	No	No	No	No

The features can be grouped into a few key categories:

• Demographic: Company Size, Industry, Job Title, Country
• Engagement: Website Visits, Webinar Attendee, Whitepaper Download, Trial Signup
• Target: Converted

Before we start exploring the data, we need to get it cleaned and prepped.

Data Preparation

Data preparation is a crucial but often overlooked part of the data science process. The goal is to get the data in a clean, consistent format so that it‘s ready for analysis. Here are the key steps we took to prep this dataset:

1. Handle missing values: Missing data can skew results and cause issues with certain analysis techniques. In this case, we had a small percentage of missing values for Company Size and Job Title. Since the amount of missing data was low, we decided to remove those records entirely. For larger amounts of missing data, imputation techniques like replacing with the mean/median or using KNN can be effective.

2. Encode categorical variables: Many models require numerical inputs, so we need to convert categorical variables to numbers. We used one-hot encoding to create binary columns for each category. For example, Industry became three columns: Industry_Technology, Industry_Healthcare, and Industry_Other.

3. Normalize numerical features: Normalization puts all the features on the same scale, which can improve model performance. We used Min-Max scaling to transform Website Visits to a range of 0 to 1.

4. Remove outliers: Outliers can have a big impact on the results, especially for parametric techniques. We plotted the distribution of Website Visits and identified a small number of extreme outliers, which we removed using the IQR method.

With the data squeaky clean, we‘re ready to start exploring!

Exploratory Analysis

The goal of EDA is to develop a deep understanding of the data and surface insights to inform modeling decisions. We‘ll start by looking at each feature individually (univariate analysis) and then move on to exploring interactions between features (multivariate analysis).

Univariate Analysis

Univariate analysis focuses on one variable at a time. We‘ll look at the distribution of each feature and identify any interesting patterns or outliers.

Company Size

Company Size is a categorical variable with three levels: Small, Medium, and Large. Here‘s a breakdown of the distribution:

Company Size	Count	Percent
Small	25000	50%
Medium	15000	30%
Large	10000	20%

The data is skewed towards smaller companies, which aligns with the company‘s target market. It will be interesting to see if there are differences in lead quality and conversion rates across company sizes.

Industry

The Industry variable captures the primary industry of the lead‘s company. It has 10 categories but the distribution is heavily concentrated in a few:

Industry	Count	Percent
Technology	10000	20%
Healthcare	7500	15%
Finance	5000	10%
Retail	5000	10%
Other	22500	45%

Over half the leads come from Technology, Healthcare, Finance, and Retail. The Other category is a catch-all for the remaining smaller industries. Industry could be a useful predictor of lead quality given the differences in buying processes and budgets across verticals.

Job Title

Job Title reflects the seniority of the lead within their company. The distribution skews towards more senior roles:

Job Title	Count	Percent
CXO	7500	15%
VP	9000	18%
Director	12500	25%
Manager	16000	32%
Other	5000	10%

CXOs, VPs, and Directors make up 58% of the leads. These decision makers likely have more buying power and authority. However, we can‘t discount managers and individual contributors who often drive software adoption from the ground up.

Country

Leads come from all over the world but are concentrated in a few key markets:

Country	Count	Percent
US	37500	75%
UK	2500	5%
Canada	2500	5%
Germany	1500	3%
France	1000	2%
India	1000	2%
Other	4000	8%

The US is by far the largest market, accounting for 75% of leads. The UK, Canada, and Germany are the next biggest at 5%, 5%, and 3% respectively. With such a heavy skew, it might make sense to build separate scoring models for US vs. International leads.

Website Visits

Website Visits is a numerical variable capturing how many times a lead visited the company website. Visits range from 0 to 100 with a very long tail:

Statistic	Value
Mean	4.5
Median	2
Min	0
Max	100
Std Dev	8.2

The distribution is highly skewed right, with a mean of 4.5 visits but a median of only 2. There are a small number of leads with extremely high numbers of visits (50+), which could be highly qualified "hot" leads. We‘ll dig into that more in the multivariate analysis.

Webinar Attendee

Webinar Attendee is a binary variable indicating whether the lead attended a company webinar:

Webinar Attendee	Count	Percent
Yes	10000	20%
No	40000	80%

Only 20% of leads attended a webinar, but this could be a strong buying signal. Webinars require a bigger time commitment than other content and reflect higher engagement.

Whitepaper Download

Another binary variable, Whitepaper Download shows if the lead downloaded a company whitepaper:

Whitepaper Download	Count	Percent
Yes	20000	40%
No	30000	60%

Whitepapers are more broadly used than webinars, with 40% of leads downloading. This likely reflects a difference in time investment – whitepapers can be quickly consumed while webinars are usually 30+ minutes. Still, a download indicates interest and could predict conversion likelihood.

Trial Signup

The last feature in our dataset is Trial Signup, which captures whether the lead signed up for a product trial:

Trial Signup	Count	Percent
Yes	5000	10%
No	45000	90%

Product trials are the highest intent activity tracked, but only 10% of leads signed up. This makes sense since trials require more effort and commitment than webinars or whitepapers. Leads that do sign up are likely very interested in purchasing.

Multivariate Analysis

While univariate analysis looks at features in isolation, multivariate analysis explores relationships and interactions between them. We‘ll use a variety of statistical techniques and visualizations to understand which feature combinations best predict conversion.

Company Size vs. Industry

Let‘s look at the interplay between Company Size and Industry. An interesting way to visualize this is with a heatmap showing the percentage of leads in each combination:

The map reveals some key patterns:

• Technology companies skew smaller, with 35% in the Small category and only 10% in Large
• Healthcare is more evenly distributed across sizes
• Finance and Retail companies tend to be larger, with around 30% each in the Large category

These differences have implications for lead scoring. Smaller tech companies are likely more receptive to inbound marketing and self-service, while larger finance/retail companies may require more high-touch sales.

Job Title vs. Country

Another interesting combination is Job Title and Country. Here‘s a stacked bar chart showing the distribution of job titles in each of the top 5 countries:

Some observations:

• The US has more CXOs (18%) and VPs (22%) than other countries
• Germany and Canada skew more heavily towards managers (45% each)
• India has an unusually high percentage of directors (40%)

This suggests leads may have different buying processes across regions. A CXO from the US might expect a more personalized sales experience, while a German manager may prefer a product-led process.

Website Visits vs. Converted

Now let‘s look at how Website Visits relates to our target variable, Converted. A box plot is a great way to compare distributions:

There‘s a clear difference between converted and non-converted leads. Leads that converted have a much higher median and IQR of website visits. Even more striking, the top 25% of converted leads had 10+ visits, while the top 25% of non-converted had only 5+. Website engagement is a powerful predictor of purchase intent.

Trial Signup vs. Webinar Attendee vs. Converted

Finally, let‘s explore how the highest intent activities, Trial Signup and Webinar Attendance, relate to conversion. A mosaic plot visualizes proportions across categorical variables:

Leads that signed up for a trial and attended a webinar had by far the highest conversion rate at 85%. Just trial signup was next highest at 50%, followed by just webinar attendance at 30%. Leads that did neither had a very low conversion rate of 5%. This suggests a scoring model that heavily weights these two behaviors.

Key Takeaways

We covered a lot of ground in this analysis! Here are the key insights to carry forward into lead scoring model development:

1. Company size and industry significantly influence buying behavior and lead quality. Scoring models should include these attributes and consider different thresholds across segments.

2. Job title and country also impact lead qualification. More senior titles from the US convert at higher rates. Consider separate models for US versus international.

3. Website visits is one of the strongest predictors of conversion likelihood. Scoring should include a website engagement component, potentially bucketed into low/medium/high activity.

4. Trial signups and webinar attendance are the highest converting activities and should be foundational in scoring models. Combining the two is especially powerful.

5. Consider enhancing the dataset with additional behavioral data like email opens, social media engagement, and chatbot interactions. These could provide even more predictive power.

Of course, this is just the beginning. These insights still need to be codified into a model, validated through testing, and embedded into marketing automation and CRM systems for real-world use. But this EDA provides a strong foundation and direction for those next steps.

It‘s also worth noting that lead scoring is not foolproof. Models can struggle with changing customer preferences, new product launches, and market shifts. Regular monitoring and model refreshes are important to maintain accuracy over time.

Additionally, scoring is just one piece of the lead qualification puzzle. To drive conversions, high-scoring leads still need effective nurturing and timely follow-ups from sales. Scoring should augment rather than replace the human element of selling.

The Future of Lead Scoring

Looking ahead, innovations in machine learning and artificial intelligence will continue to advance lead scoring. ML models can analyze vast amounts of structured and unstructured data to find non-intuitive patterns. Deep learning image recognition models could score leads based on their social media photos and posts. Natural language processing could gauge intent based on the sentiment of online conversations.

The possibilities are exciting but come with a note of caution. Advanced techniques require sizable investments in data infrastructure, modeling talent, and change management. Companies must weigh the marginal benefits of complexity against the proven results of more straightforward scoring approaches. An effective but simple model beats a sophisticated but cumbersome one every time.

Conclusion

This deep dive into lead scoring exploratory data analysis demonstrates the power of letting the data tell a story. With some basic statistical tools and a curious mindset, we uncovered clear patterns around which leads are most likely to convert. These insights pave the way for building a lead scoring model that drives real business results.

But EDA is just one chapter of the story. The most successful lead scoring initiatives combine thoughtful data analysis, carefully crafted models, intuitive user experiences, and ongoing optimization. Marketers should think holistically about how scoring fits into their broader demand generation engine.

One thing is certain – in a world of constrained budgets and rising customer expectations, robust lead qualification is becoming a competitive necessity. Companies that crack the code on quickly identifying high-value prospects and delivering targeted experiences will rise to the top. Effective exploratory data analysis is a crucial starting point on that journey.