Overcoming Position Bias in Recommendation and Search Systems: An AI/ML Perspective

Introduction

Modern recommendation and search systems heavily rely on user interaction data like clicks and views to optimize their rankings. The most common approach is to train machine learning models to predict the probability of a user engaging with an item based on various relevance features. The items are then ranked according to these predicted probabilities.

However, this interaction data is plagued by various biases, most notably position bias. Users tend to interact more with items ranked highly in the list, regardless of their actual relevance or quality. This leads to a feedback loop where the most clicked items keep getting shown at the top, while other potentially relevant items are buried. Over time, this degrades the quality and diversity of the recommendations.

To build truly user-centric ranking systems, we must account for position bias and other confounding factors in user interaction data. This post will dive deep into the issue of position bias and present state-of-the-art techniques for mitigating it, with a focus on the position-aware learning approach. We‘ll cover the latest research advancements and share best practices from real-world applications at scale.

Measuring Position Bias

The first step in addressing position bias is quantifying its impact. We can measure the degree of bias by comparing the click-through rate (CTR) of the same item in different positions. If the CTR varies significantly by position, this indicates the presence of bias.

One common technique is to run randomized experiments on a small percentage of live user traffic. The idea is to randomly shuffle the ranking of items shown to the user to remove the effect of position. We can then calculate the CTR for each position and compare it to the global average.

For example, consider the following results from a randomization experiment on a search query for "best laptops":

Position	Impressions	Clicks	CTR
1	10,000	3,500	35.00%
2	10,000	2,800	28.00%
3	10,000	2,000	20.00%
4	10,000	1,200	12.00%
5	10,000	500	5.00%

The global average CTR across all positions is 20%. However, we observe a strong position bias – the 1st position item has a 75% higher CTR compared to the average. This multiplicative factor is known as the position bias ratio.

An alternative approach is to analyze historical click logs and segment queries into groups with similar ranking permutations. We can estimate the per-position CTR within each group and aggregate the results. This method leverages naturally occurring variations in rankings but may suffer from selection bias, as more relevant items tend to be ranked higher.

Inverse Propensity Weighting

A popular technique for debiasing user interaction data is inverse propensity weighting (IPW). IPW assigns a weight to each click based on the probability that the user examined the clicked item. The intuition is that a click on a lower-ranked item is more informative because the user likely scanned past other items to reach it.

IPW relies on the following click modelization:

$P(click) = P(examine) * P(relevant)$

In other words, the probability of a click is the product of two factors:

1. The probability that the user examined the item
2. The relevance or quality of the item

The position bias comes from the examination probability $P(examine)$, which depends on the item‘s rank. If we can estimate this propensity, we can isolate the true relevance:

$P(relevant) = \frac{P(click)}{P(examine)}$

There are several methods for estimating the propensities:

1. Randomization experiments like the one shown above to measure the position bias ratio at each rank.
2. User browsing models that predict the examination probability based on rank and other behavioral signals like dwell time and scroll depth. The most common is the cascade model, which assumes users scan the list from top to bottom until they find a relevant item.
3. Intervention harvesting that uses naturally occurring variations in ranking permutations to estimate propensities without randomization.

The main drawback of IPW is that it requires some form of randomization or intervention in the live ranking system, which may degrade the user experience. It also fails to capture more complex position effects beyond the simple cascade assumption.

Position-Aware Learning

A more recent and promising approach is position-aware learning (PAL), which directly incorporates the position signal into the ranking model. Instead of debiasing the interaction data as a preprocessing step, PAL lets the model learn the effect of position during training and correct for it at inference time.

The key idea is to include the position as an input feature along with the regular query-item relevance features. For a search ranking problem, the input features for an item might be:

• BM25 text match score between query and item title
• Number of query terms in item description
• Item popularity score
• Item position (e.g., 1, 2, 3)

During training, the model learns to predict the observed CTR conditioned on both the relevance features and the position. Importantly, at inference time, we intervene by fixing the position feature to a constant value (usually the average position) for all items. This removes the influence of position and puts the items on a level playing field.

Essentially, PAL captures the following causal relationship:
$CTR = f(relevance, position)$

By learning the function $f$ and manipulating the position variable, we can estimate the unbiased relevance of each item. PAL has several advantages over IPW:

• It doesn‘t require any modifications to the logging policy or live ranking system. The intervention happens purely at training time.
• It can learn more complex position effects like presentation bias that depends on UI elements.
• It‘s a general framework that can be applied to different model architectures like neural networks and tree-based models.

PAL has been successfully deployed in production systems at companies like Google [1], Netflix [2], and Spotify [3]. It powers Google‘s flagship search ranking system [4] and is used extensively in Gmail‘s email prioritization [5]. Open-source libraries like TensorFlow Ranking [6] and LightGBM [7] also provide easy-to-use PAL implementations.

Research Frontiers

PAL falls under the general umbrella of counterfactual learning and causal inference, which aim to estimate treatment effects from observational data [8]. Recent advancements in this area have led to improved debiasing methods for implicit feedback data in ranking and recommendations.

One line of work focuses on doubly robust estimators that combine the benefits of IPW and model-based imputation [9]. These estimators are more resilient to model misspecification and can achieve lower variance.

Another promising direction is to learn invariant predictors that are robust to distributional shifts caused by position bias [10]. The idea is to penalize the model for learning spurious correlations between the position and target variable. This can be formulated as a domain adaptation or distribution matching problem.

There have also been attempts to apply ideas from causal representation learning to debias recommender systems [11]. The goal is to learn representations of users and items that are invariant to biases like popularity and position. This allows for more stable and generalizable recommendations.

On the theoretical side, researchers have analyzed the sample complexity and generalization bounds of PAL and related methods [12]. These results provide guidance on how much interaction data is needed to learn unbiased relevance estimators under different assumptions.

Practical Guidance

Here are some best practices and considerations for implementing PAL in real-world ranking systems:

1. Feature Engineering: Include position as a categorical feature, with each unique position value as a separate category. You can also add other position-related features like page number and contextual information about the UI layout. Be sure to normalize these features to avoid scale issues.

2. Model Architecture: PAL can be applied to any differentiable model architecture like logistic regression, feed-forward neural networks, and gradient boosted trees. The key is to include the position feature in the input layer and fix its value at inference time. For neural networks, you can use an embedding layer to learn a dense representation of each position.

3. Hyperparameter Tuning: The optimal PAL model may require different hyperparameters than the regular model without position features. In particular, you may need to adjust the regularization strength to prevent overfitting on the position signal. Use a held-out validation set to tune the hyperparameters and evaluate the debiased performance.

4. Online Evaluation: The ultimate test of a debiased ranking model is its impact on online user engagement metrics. Run A/B experiments to compare the PAL model against the baseline model without position features. Look at metrics like CTR, conversion rate, and user satisfaction scores. Be sure to run the experiment long enough to capture any long-term effects.

5. Monitoring and Updating: Position bias can change over time as users adapt to the new ranking system. It‘s important to continuously monitor the performance of the PAL model and retrain it periodically with fresh data. You can also use online learning techniques to update the model parameters in real-time based on user feedback.

Here‘s an example of how to apply PAL to a simple logistic regression model using Python and the TensorFlow Ranking library:

import tensorflow as tf
import tensorflow_ranking as tfr

# Define input features
query_features = {
    "query_tokens": tf.feature_column.categorical_column_with_vocabulary_list(
        "query_tokens", vocabulary_list=["laptop", "computer", "notebook"]),
    "item_title_tokens": tf.feature_column.categorical_column_with_vocabulary_list(
        "item_title_tokens", vocabulary_list=["macbook", "thinkpad", "xps"]),
}

# Define position feature
position_feature = tf.feature_column.categorical_column_with_identity(
    "position", num_buckets=10)

# Combine features
ranking_features = tf.feature_column.indicator_column(
    tf.feature_column.shared_embeddings(
        [query_features, position_feature], 
        dimension=16, 
        combiner=‘mean‘))

# Define model
ranking_model = tf.estimator.LinearClassifier(
    feature_columns=[ranking_features],
    optimizer=tf.compat.v1.train.FtrlOptimizer(learning_rate=0.1))

# Train model
train_input_fn = tf.compat.v1.estimator.inputs.numpy_input_fn(
    x={
        "query_tokens": train_queries, 
        "item_title_tokens": train_item_titles,
        "position": train_positions
    },
    y=train_labels,
    batch_size=32,
    num_epochs=10,
    shuffle=True)

ranking_model.train(input_fn=train_input_fn)

# Evaluate model
eval_input_fn = tf.compat.v1.estimator.inputs.numpy_input_fn(
    x={
        "query_tokens": test_queries,
        "item_title_tokens": test_item_titles,
        "position": test_positions
    },
    y=test_labels,
    num_epochs=1,
    shuffle=False)

# Fix position to average value for unbiased evaluation
fixed_position = np.mean(test_positions)
unbiased_eval_input_fn = tf.compat.v1.estimator.inputs.numpy_input_fn(
    x={
        "query_tokens": test_queries,
        "item_title_tokens": test_item_titles,
        "position": fixed_position
    },
    y=test_labels,
    num_epochs=1,
    shuffle=False)

biased_metrics = ranking_model.evaluate(input_fn=eval_input_fn)
unbiased_metrics = ranking_model.evaluate(input_fn=unbiased_eval_input_fn)

This code snippet demonstrates the key steps in applying PAL:

1. Define the query-item relevance features and position feature
2. Combine the features into a single feature column for input to the model
3. Train the model on the full set of features
4. Evaluate the model on held-out data with:
   • Original position feature for measuring biased performance
   • Fixed position feature set to mean value for measuring unbiased relevance

By comparing the biased and unbiased evaluation metrics, we can quantify the impact of position bias and assess the effectiveness of the PAL debiasing.

Conclusion

Position bias is a major challenge in learning from user interaction data in ranking and recommendation systems. If left unchecked, it can lead to suboptimal model performance and a poor user experience. While inverse propensity weighting is a classical approach to debiasing, it has several limitations and may require intrusive interventions.

Position-aware learning offers a more flexible and robust solution by directly modeling the effect of position in the ranking objective. PAL has proven to be effective in reducing position bias and improving ranking quality in both academic research and industry applications. By including position as an input feature and fixing its value at inference time, PAL allows the model to learn the true relevance of items while controlling for positional effects.

PAL is part of a broader family of counterfactual learning techniques that use causal inference to estimate unbiased treatment effects from observational data. This is an active area of research with many promising advancements in recent years, such as doubly robust estimators, invariant learning, and causal embeddings. As the field progresses, we can expect to see more principled and scalable methods for debiasing implicit feedback data.

To apply PAL in practice, it‘s important to carefully design the position features, choose an appropriate model architecture, and tune the hyperparameters on a validation set. Online experimentation is crucial for measuring the real-world impact of the debiased model on user engagement metrics. Continuous monitoring and updating of the model is also necessary to adapt to changes in user behavior and position bias over time.

In conclusion, position bias is a critical issue to address in building fair and effective ranking systems. Position-aware learning is a powerful tool in the AI/ML practitioner‘s toolkit for mitigating this bias and learning unbiased relevance models. By combining PAL with other techniques like doubly robust estimation and invariant learning, we can develop more robust and generalizable ranking systems that truly put the user first.