A Data-Driven Look at The Big Bang Theory: EDA and Recommendation Systems

The Big Bang Theory is one of the most popular and long-running sitcoms of all time. Over the course of 12 seasons and 279 episodes from 2007 to 2019, the show followed the lives of a group of geeky friends in Pasadena, California. In this article, we‘ll take a data-driven look at The Big Bang Theory, performing exploratory data analysis and building a recommendation system using a dataset of the show‘s episodes.

Through this analysis, we‘ll uncover insights about patterns and trends across the series‘ run, as well as create a model to suggest similar episodes for viewers to watch next. Whether you‘re a fan of the show or simply interested in seeing data science techniques applied to pop culture data, this article will walk through the end-to-end process in an informative and engaging way. Let‘s dive in!

Preparing the Dataset

The first step is acquiring a suitable dataset. Fortunately, there are several datasets about The Big Bang Theory publicly available online. For this analysis, we‘ll use a dataset from data.world user priyankad0993 which contains information on every episode of the series including:

• Season and episode number
• Episode title
• Air date
• Rating from IMDB users
• Total votes
• Brief plot description

After downloading the dataset in CSV format, we can load it into a pandas DataFrame in a Jupyter notebook:

import pandas as pd

df = pd.read_csv(‘BigBangTheory_IMDB.csv‘) 

Let‘s take a look at the first few rows:

df.head()

And get a summary of the DataFrame:

df.info()

This shows we have data on 279 episodes across 8 columns, with no missing values. The data types look good as well, with seasons and episodes stored as integers, ratings and votes as floats, and everything else as strings.

Before analyzing further, let‘s do some light data cleaning. First we‘ll check for any duplicate rows:

df.duplicated().sum()  

Looks like there are no duplicates. We already confirmed there are no missing values. The only other cleanup to do is some formatting of the season and episode columns. Let‘s prepend an "S" to the season and "E" to the episode number to make them more readable:

df[‘Season‘] = ‘S‘ + df[‘Season‘].astype(str)
df[‘EpisodeNum‘] = ‘E‘ + df[‘EpisodeNum‘].astype(str)

Now the DataFrame is ready for exploration!

Exploratory Analysis

With the data loaded and cleaned, we can start looking for interesting patterns and insights. Some key questions to investigate:

• How many episodes were in each season? Were any seasons longer or shorter than others?
• Which episodes were the highest and lowest rated by IMDB users?
• Did average episode rating change over the course of the series?
• What was the overall average rating for the entire show?

To make the findings more visual, we‘ll use Matplotlib and Seaborn to create charts and graphs. Let‘s import those libraries and configure some style settings:

import matplotlib.pyplot as plt
import seaborn as sns

%matplotlib inline
sns.set_style(‘darkgrid‘) 
plt.rcParams[‘figure.figsize‘] = (9, 5)

First up, let‘s count the number of episodes in each season and plot it as a bar chart:

season_counts = df[‘Season‘].value_counts().sort_index()

sns.barplot(x=season_counts.index, y=season_counts);
plt.title(‘Episodes per Season‘);

Looks like most seasons had either 23 or 24 episodes, which is typical for a network TV show. The exception is the 12th and final season which was a bit shorter at 17 episodes.

Next let‘s find the top and bottom rated episodes according to IMDB users. We can use nlargest and nsmallest to get the episodes with the highest and lowest scores:

top10_episodes = df.nlargest(10, ‘Rating‘)
top10_episodes[[‘Season‘, ‘EpisodeNum‘, ‘Title‘, ‘Rating‘]]

bottom10_episodes = df.nsmallest(10, ‘Rating‘)  
bottom10_episodes[[‘Season‘, ‘EpisodeNum‘, ‘Title‘, ‘Rating‘]]

Interestingly, many of the top episodes are from the later seasons, while many of the lower rated ones are from earlier on. Perhaps the show hit its stride after a few seasons.

To dig into ratings a bit more, let‘s calculate the average episode rating for each season and chart that as a line graph:

season_ratings = df.groupby(‘Season‘)[‘Rating‘].mean()

sns.lineplot(x=season_ratings.index, y=season_ratings);  
plt.title(‘Average Episode Rating by Season‘);

This reveals an upward trend in episode ratings as the series progressed, going from an average around 8.0 in season 1 to 8.4 by season 12. The show seemed to steadily improve over time according to viewers.

Finally, we can easily calculate the overall series average:

series_avg_rating = df[‘Rating‘].mean()
print(f‘Average series rating: {series_avg_rating:.2f}‘)  

An impressive 8.19 overall! The Big Bang Theory was clearly a very well-liked and consistently great show as judged by its audience.

Building a Recommendation Model

In addition to analyzing existing data on the show, we can also use it to power a recommendation system to suggest episodes for viewers to watch next. There are a variety of approaches to building recommenders, but since we have brief plot descriptions for each episode, we can create a content-based model.

The idea is to vectorize the text descriptions, calculating their similarity to one another, and when a user watches one episode, suggest the most similar unwatched episodes. To convert the descriptions to numerical vectors, we‘ll use scikit-learn‘s CountVectorizer:

from sklearn.feature_extraction.text import CountVectorizer

vectorizer = CountVectorizer(stop_words=‘english‘)
vectors = vectorizer.fit_transform(df[‘Plot‘])  

This tokenizes each plot, counts the frequencies of each word, and filters out common stop words. The output is a sparse matrix of episodes and their word counts.

With the episodes vectorized, we can calculate the pairwise cosine similarity between their vectors:

from sklearn.metrics.pairwise import cosine_similarity

similarity_matrix = cosine_similarity(vectors)

Now for a given episode, we can find the top N most similar episodes by sorting the similarity scores. Let‘s write a function to return the most similar episodes for a query:

def get_recommendations(episode, n=5):
  episode_idx = df[df[‘Title‘] == episode].index[0]
  episode_similarities = sorted(list(enumerate(similarity_matrix[episode_idx])), 
                                key=lambda x: x[1], reverse=True)

  print(f‘Because you watched {episode}:‘)
  for idx, score in episode_similarities[1:n+1]:
    print(f"You might like: {df.loc[idx, ‘Title‘]} (similarity: {score:.2f})")  

This takes in an episode title, looks up its index in the DataFrame, gets its corresponding row from the similarity matrix, sorts the scores to find the top N matches, and prints out the titles.

Let‘s test it with an example episode from season 9:

test_episode = ‘The Platonic Permutation‘
get_recommendations(test_episode)

Because you watched The Platonic Permutation:
You might like: The Mystery Date Observation (similarity: 0.32)  
You might like: The Earworm Reverberation (similarity: 0.29)
You might like: The Matrimonial Momentum (similarity: 0.27)
You might like: The 2003 Approximation (similarity: 0.26)
You might like: The Emotion Detection Automation (similarity: 0.24)

Not bad! The model seems to be picking up on similarities between episodes based on their descriptions. This could serve as a good basis for a more robust recommender, perhaps incorporating user ratings, watch history, and more.

Further Ideas and Next Steps

There are many ways to extend this analysis and recommender model:

Using the ‘Votes‘ column: We didn‘t incorporate the number of user votes per episodes in this project, just the average rating. More votes generally means a more reliable score – a 9.0 rated episode with 5000 votes is likely better than a 9.5 with only 100 votes. A weighted rating could lead to better recommendations.

Enhancing the recommender: Our simple content-based model only used plot descriptions, but there are opportunities to fold in more information to improve suggestions, such as:

• Episode‘s season (e.g. recommend from same or adjacent season)
• Episode‘s rating (e.g. don‘t recommend poorly rated episodes)
• User‘s watch history (e.g. don‘t recommend already seen episodes)

Exploring other datasets: The techniques covered here can be applied to all sorts of other TV shows, movies, books, music, and more. There are tons of fascinating pop culture datasets to analyze and build models with. Some places to look for data:

• Streaming service APIs (e.g. Spotify, Netflix, etc.)
• Open data repositories (e.g. Kaggle, data.world, etc.)
• Web scraping sites like IMDB, Rotten Tomatoes, Goodreads, etc.

I encourage you to find a dataset that interests you and see what insights you can uncover! The domain of data science is vast and varied, and it‘s a blast applying its tools and techniques to areas you‘re passionate about.

Conclusions

In this article, we took a fun and informative look at The Big Bang Theory through the lens of data. Performing exploratory analysis, we discovered:

• The series had a very consistent 23-24 episodes per season across its 12 year run
• User ratings trended upwards over time, with many top episodes in later seasons
• The show maintained an impressive 8.19 average rating throughout

We also built a basic content-based recommendation model using episode plot descriptions to suggest similar episodes to viewers. While just a starting point, it demonstrated the potential for data-driven personalized TV recommendations.

There are endless opportunities to conduct similar analyses on other shows and cultural works, deriving fascinating insights and building useful predictive models. And the dataset and code here can be used as a template to jump start future projects.

I hope this article has piqued your curiosity to explore pop culture through data, and given you some tools and ideas to do so. With so much data out there, there‘s never been a better time to practice data science with the movies, shows, books, and music you love. Happy analyzing!