Building a Product Recommendation System using Word2vec in Python
Introduction
In today‘s digital age, recommendation systems have become ubiquitous and indispensable for many online businesses. From Amazon to Netflix to Spotify, recommender systems power personalized user experiences and help customers discover new and relevant products or content. Building effective recommendation systems is both a science and an art, requiring a blend of machine learning techniques, domain knowledge, and an understanding of human behavior and preferences.
In this tutorial, we‘ll walk through how to build a product recommendation system using a technique called word2vec, borrowed from the field of natural language processing (NLP). While traditionally used for learning semantic relationships between words in a corpus of text, we‘ll show how word2vec can be creatively repurposed to learn latent relationships between products based on customer purchase histories. By the end, you‘ll gain both a conceptual understanding of the inner workings of word2vec as well as hands-on experience leveraging it to build a product recommender in Python. Let‘s dive in!
Overview of Recommendation Systems
At a high level, recommendation systems aim to predict and recommend the most relevant items to a particular user based on the user‘s previous behavior and preferences. The items can span a wide range, including products, articles, songs, movies, restaurants, online courses, and more, depending on the application domain. Recommenders unlock substantial business value in the form of increased sales and revenue, improved user satisfaction and loyalty, and better inventory and resource planning.
Broadly speaking, recommendation systems can be categorized into three main types:
- Content-based: recommend items similar to what a user has liked in the past based on intrinsic characteristics of the items
- Collaborative filtering: recommend items that users with similar tastes and preferences have liked in the past
- Hybrid: combine both content-based and collaborative filtering approaches
Some common approaches and algorithms used to build recommenders include:
- Matrix factorization
- Deep learning
- k-Nearest Neighbors
- Association rule mining
- Probabilistic graphical models
In this tutorial, we‘ll focus on using word2vec, a technique grounded in both matrix factorization and deep learning, to build a recommender system for an online retail store.
Word2vec Primer
Word2vec is a popular algorithm from natural language processing (NLP) for learning vector representations of words, called "word embeddings". It was originally introduced by a team at Google led by Tomas Mikolov in 2013. The key idea behind word2vec is to learn a dense, low-dimensional vector for each word based on its usage context in a corpus of text. Words that are used in similar contexts will have vectors that are close together in the embedding space.
There are two main model architectures for learning word vectors in word2vec:
- Continuous Bag-of-Words (CBOW): predict a target word given its surrounding context words
- Continuous Skip-gram: predict the surrounding context words given a target word
In both architectures, word2vec learns the word embeddings by training a shallow neural network to perform these prediction tasks and backpropagating the errors to update the embeddings.One of the key insights behind word2vec is that the learned word vectors encode semantic meaning and relationships. For example, the embedding for "king" minus "man" plus "woman" results in a vector very close to "queen". This property can be used for reasoning by analogy and uncovering latent semantic connections.
Preparing Online Retail Data
To build our product recommendation system, we‘ll use a public dataset of ~540K transactions from a UK-based online retail store, spanning the period 12/01/2010 to 12/09/2011. Each transaction includes the following fields:
- Invoice number
- Stock code
- Product description
- Quantity
- Invoice date
- Customer ID
Our goal will be to recommend relevant products to a user based on their past purchase history. To do this, we‘ll train a word2vec model on sequences of products purchased by each customer.
First, let‘s load the data into a Pandas DataFrame and do some basic preprocessing:
import pandas as pd
df = pd.read_excel(‘Online Retail.xlsx‘)
df.head()InvoiceNo StockCode Description Quantity \ 0 536365 85123A WHITE HANGING HEART T-LIGHT HOLDER 6 1 536365 71053 WHITE METAL LANTERN 6 2 536365 84406B CREAM CUPID HEARTS COAT HANGER 8 3 536365 84029G KNITTED UNION FLAG HOT WATER BOTTLE 6 4 536365 84029E RED WOOLLY HOTTIE WHITE HEART. 6InvoiceDate UnitPrice CustomerID Country0 2010-12-01 08:26:00 2.55 17850.0 United Kingdom
1 2010-12-01 08:26:00 3.39 17850.0 United Kingdom
2 2010-12-01 08:26:00 2.75 17850.0 United Kingdom
3 2010-12-01 08:26:00 3.39 17850.0 United Kingdom
4 2010-12-01 08:26:00 3.39 17850.0 United KingdomLet‘s check for any missing values:
df.isnull().sum()InvoiceNo 0 StockCode 0 Description 1454 Quantity 0 InvoiceDate 0 UnitPrice 0 CustomerID 135080 Country 0 dtype: int64Since there are some missing values for CustomerID, we‘ll remove those rows:
df.dropna(subset=[‘CustomerID‘], inplace=True)Next, we‘ll group the transactions by CustomerID and create a list of StockCodes purchased for each customer:
purchases = df.groupby(‘CustomerID‘)[‘StockCode‘].apply(list).reset_index() purchases.head()CustomerID StockCode 0 12346.0 [85123A, 71053, 84406B, 84029G, 84029E, 22752,... 1 12347.0 [85123A, 84077, 84879, 22411, 21731, 22623, 2... 2 12348.0 [84879, 15056BL, 15056N, DOT] 3 12349.0 [21731] 4 12350.0 [22650, 22407]We now have our customer "documents" ready in a format compatible with word2vec – each customer is represented by a sequence of product "words".
Training a Word2vec Model
With our data prepared, we‘re ready to train our word2vec model. We‘ll use the popular Gensim library in Python, which has an optimized implementation of word2vec.
First, let‘s split our data into a train and validation set:
from sklearn.model_selection import train_test_split train, valid = train_test_split(purchases, test_size=0.2, random_state=2023)Now, let‘s instantiate our word2vec model and train it on the purchase sequences:
from gensim.models import Word2Vec model = Word2Vec(sentences=train[‘StockCode‘], vector_size=100, window=5, min_count=1, workers=4) model.train(train[‘StockCode‘], total_examples=len(train), epochs=10)We set
vector_sizeto 100, which means each product will be represented by a dense vector of 100 floating point numbers. Thewindowparameter controls the size of the surrounding context for each target word.min_countsets the minimum frequency threshold for a word to be included in the vocabulary. We train the model for 10 epochs on the sequences.After training, we can access the learned product embeddings through
model.wv:product_vectors = model.wv print(len(product_vectors))3684Our model has learned embeddings for 3684 unique products. Let‘s spot check a few similarity examples:
product_vectors.most_similar(‘84879‘)[(‘84991‘, 0.877117395401001), (‘22197‘, 0.8618917465209961), (‘22428‘, 0.8580918312072754), (‘22424‘, 0.8369876742362976), (‘22427‘, 0.8275295495986938)]The model returns the top 5 most similar products to "84879" based on cosine similarity of the vectors. Not bad!
Visualizing Product Embeddings
Since our product embeddings live in a 100-dimensional space, we can‘t visualize them directly. However, we can apply dimensionality reduction techniques like PCA or t-SNE to project them into a 2D space for visualization purposes.
Let‘s use t-SNE from scikit-learn:
from sklearn.manifold import TSNE tsne = TSNE(n_components=2, verbose=1, random_state=2023) X_2d = tsne.fit_transform(product_vectors.vectors)Now, let‘s plot the 2D embeddings with Matplotlib:
import matplotlib.pyplot as plt plt.figure(figsize=(12, 8)) plt.scatter(X_2d[:, 0], X_2d[:, 1]) plt.title(‘2D Projection of Product Embeddings‘) plt.show()
The plot shows some local clustering structure, where similar products are embedded close together in the space. We can even annotate a few points with the actual product descriptions to interpret the semantic relationships the model has learned.
Generating Product Recommendations
Now for the fun part – using our trained embeddings to recommend products! The general approach will be:
- Given a user‘s purchase history, represent it by aggregating (summing or averaging) the vectors of the purchased products
- Find the top k most similar products to this aggregated user vector
Let‘s write a helper function to get the top k most similar products to a given product:
def get_similar_products(product_id, k=5): try: similar_products = product_vectors.most_similar(product_id, topn=k) return similar_products except KeyError: return []To recommend products for a user, we‘ll aggregate their purchase history vector and then find the most similar products to this vector:
def recommend_products(purchase_history, k=5, aggregation=‘mean‘): product_vectors = [model.wv[product_id] for product_id in purchase_history if product_id in model.wv] if len(product_vectors) == 0: return [] if aggregation == ‘mean‘: user_vector = np.mean(product_vectors, axis=0) elif aggregation == ‘sum‘: user_vector = np.sum(product_vectors, axis=0) else: raise ValueError(‘Unknown aggregation type: {}‘.format(aggregation)) return model.wv.most_similar(positive=[user_vector], topn=k)Let‘s test it out on a sample user from our validation set:
sample_user = valid.iloc[0] print(‘User ID:‘, sample_user[‘CustomerID‘]) print(‘Purchase History:‘, sample_user[‘StockCode‘]) print(‘Recommendations:‘) for product_id, score in recommend_products(sample_user[‘StockCode‘]): print(‘{}: {:.3f}‘.format(product_id, score))User ID: 15098.0 Purchase History: [22720, 21786, 22419, 22067, 85116, 22065, 22908, 22357]Recommendations:
22969: 0.868 22139: 0.864 85064: 0.837 22423: 0.836 22111: 0.836Our model recommends 5 new products that this user may be interested in based on the learned product similarities. In practice, you‘d want to filter out products the user has already purchased before making recommendations.
Extensions and Next Steps
Congrats on making it this far! Let‘s discuss some potential extensions and improvements to our basic recommender system:
- Experiment with different word2vec hyperparameters (vector size, context window, etc.)
- Try alternate product aggregation schemes (weighted average, exponential decay, etc.)
- Combine product embeddings with user embeddings for more personalized recommendations
- Integrate side information (product metadata, user demographics, etc.) into the model
- Explore other embedding approaches (e.g. GloVe, fastText)
- Evaluate recommendations in an online A/B test
The complete code for this tutorial is available on GitHub. Feel free to use it as a starting point for your own projects.
Conclusion
In this tutorial, we covered how to build a product recommendation system using word2vec embeddings in Python. The key idea was to frame the problem in a way that allowed us to repurpose word2vec, an algorithm originally designed for learning word embeddings from natural language, to instead learn product embeddings from sequences of user purchases.
Through code examples, we demonstrated the end-to-end process of preparing the online retail dataset, training a word2vec model, visualizing the learned product embeddings, and ultimately leveraging them to generate personalized product recommendations for users based on their individual purchase histories.
While this approach is by no means the only way to build a product recommender system, it highlights the power and flexibility of embeddings as a general purpose tool for representation learning, as well as how techniques from one domain (like NLP) can often be creatively adapted to solve problems in another.
I hope this has been an informative and practical introduction to recommendation systems for you. The complete code is available, so I encourage you to try running and extending it yourself. Best of luck on your learning journey!
