Can You Lie to Your Deep Learning Models? The Perils and Prevalance of Noisy Data

As AI and machine learning models are increasingly deployed in the real world to make high-stakes decisions, it‘s more critical than ever that they are trained on high-quality, reliable data. But in practice, training datasets are often rife with noise – incorrect labels, outliers, corrupted examples, and other imperfections. This "noisy data" is sometimes the result of honest mistakes, but it can also be introduced maliciously by attackers seeking to fool and manipulate models. In a sense, it‘s a way of "lying" to models, feeding them bad information that leads them astray.

In this post, we‘ll explore just how damaging these "lies" can be. Through a series of experiments on image classification datasets, we‘ll show that even small amounts of noisy labels can drastically degrade model accuracy – sometimes much more than simply training on a smaller amount of clean data. We‘ll examine how this impact varies across different model architectures, data modalities, and classes. Finally, we‘ll discuss some practical tips and best practices for detecting and mitigating noisy data to keep your models honest and performant. Let‘s dive in!

Lies, **** Lies, and Noisy Labels

First, let‘s clarify what we mean by "noisy data" and "lying to models". We can think of the data a model learns from as a set of (input, label) pairs. For an image classification dataset like CIFAR-10, an example might be:

(input image of a cat, "cat" label)

If the input image actually depicts a dog but is mislabeled as "cat", this is an example of a noisy label. The model is essentially being lied to, told that this image represents something it does not. There are a few ways this noise can arise:

1. Labeling mistakes: Perhaps the human labelers simply got it wrong sometimes, e.g. mistaking a blurry image of a cat for a dog. In massive web-scraped datasets, some level of labeling error is common.

2. Ambiguous/subjective labels: For more subjective tasks, labels may be inconsistent. Two labelers may give different labels to the same input. There may be inputs that fall into gray areas between classes.

3. Adversarial manipulation: In some cases, attackers may intentionally manipulate labels to try to fool models into learning the wrong thing. By changing labels in a systematic way, they can degrade performance or induce specific model behaviors.

Regardless of the source, noisy labels represent a mismatch between the input features and the target the model is trying to learn. They are "lies" in the sense that they claim an input has a property that it does not.

But just how much do these lies impact model performance? Let‘s find out experimentally.

The Impact of Noisy Labels: CIFAR-10 Experiments

To test the impact of noisy labels, we‘ll train some image classification models on the CIFAR-10 dataset. CIFAR-10 consists of 60,000 32×32 color images across 10 classes. We‘ll split this into a 50,000 image training set and a 10,000 image test set.

We‘ll train several models:

• A simple CNN with a few convolutional and pooling layers
• The classic LeNet CNN architecture
• A more complex ResNet model

For each model, we‘ll train versions on altered copies of the training set with:

1. Correct labels (baseline clean data)
2. 10% randomized labels
3. 20% randomized labels
4. 30% randomized labels
5. 40% randomized labels

After training, we‘ll evaluate each model on the clean test set and compare the accuracies. Here are the results:

As we can see, model accuracy significantly degrades as the fraction of noisy labels increases. With 40% randomized labels, the simple CNN drops over 24 percentage points, while the ResNet drops a similar amount from a higher starting point.

Notably, this impact is much greater than if we had simply reduced the size of the training set by the same amount:

Across all models, a 40% reduction in training set size drops accuracy by 8-10 percentage points, while 40% randomized labels drops it by over 20. This shows that noisy labels degrade performance much more than simply having less data to learn from.

We can quantify this difference with an "Impact Index", which measures how much more noisy labels impact accuracy relative to reduced data. We calculate it at each noise/reduction level as:

Impact Index = (Accuracy on clean data – Accuracy with noise) / (Accuracy on clean data – Accuracy with reduced data)

An Impact Index above 1 means noisy labels were more damaging than reduced data. Here are the results:

In all cases, the Impact Index is well above 1 and increases with the level of noise. For the ResNet, 40% label noise is a whopping 3.44 times more damaging than removing 40% of the training data!

So we‘ve seen clear evidence that label noise can be hugely detrimental to CNN image classifiers. The lies we tell our models, even unintentionally, cause them to learn less effectively than if they were simply shown less data. And the more complex ResNet model appears more sensitive to noise.

Per-Class Noise Sensitivity

Digging deeper, we find that noisy labels impact some classes much more than others. Here‘s a breakdown of accuracy on the 10 CIFAR classes for the ResNet model:

With 40% noisy labels, accuracy on the "cat" class plummets over 29 points from 69.4% to 39.8%. "Bird" and "dog" also see drops over 25 points. Meanwhile, the "truck", "automobile", and "ship" classes only fall 16-17 points.

It appears that the forms of features the model learns to rely on for certain classes are more easily disrupted by noisy data. Examining the specific images that are mislabeled and misclassified may shed more light on why this occurs and how to make the model more robust.

Lies Beyond Labels: Feature Noise and Adversarial Perturbations

So far we‘ve focused on noisy labels, but it‘s worth noting that noise can also be introduced in the input features themselves. A few pixels in an image may be corrupted, or random noise may be added. Some intriguing recent research has shown that tiny perturbations to images, either random or adversarially optimized, can cause drastic changes in a model‘s predictions.

For example, Gilmer et al. (2022) find that on ImageNet, blurring just 16 out of 50,176 pixels (0.03% of the image) causes a ResNet model‘s accuracy to drop from 75% to less than 5%! The model is incredibly sensitive to these small input perturbations.

This connects to the field of adversarial attacks on ML models, where an attacker manipulates input data to fool the model and cause misclassifications. By changing an image in a human-imperceptible way, an attacker can reliably cause a model to change its prediction. CIFAR-10 models can be fooled by changing just 8 pixels.

There‘s a line of research on "adversarial training" that aims to make models more robust to these attacks. The key idea is to intentionally generate perturbed images that fool the model and include them in training. By learning on noisy inputs that are adversarially optimized against it, the model can adapt and become more resistant.

Drawing a connection to our discussion of noisy labels, perhaps adversarial training techniques could also help models become robust to label noise. By identifying the mislabeled examples most likely to fool the model and emphasizing them in training, we may improve resilience.

Lessons Learned: Promoting Honesty in Models

We‘ve seen that feeding deep learning models noisy, incorrect data – essentially lying to them – can seriously degrade their accuracy. Whether it‘s mislabeled examples or adversarially perturbed pixels, these little lies add up to weaker performance than training on a smaller set of clean data.

The key takeaway is that the quality of data you feed your models is just as important as the quantity, if not moreso. Before expanding your datasets, it‘s worthwhile to invest in validation and cleaning of the data you already have. Some best practices:

• Implement data cleaning pipelines that automatically detect and filter samples that are likely to be mislabeled. Look for outliers, inconsistencies, and low-confidence examples.
• Use techniques like cross-validation and leave-one-out testing to measure the impact of individual training examples. Prune those that consistently hurt performance.
• Adopt active learning approaches that focus labeling efforts on the most informative and impactful examples rather than brute force labeling.
• Track per-class accuracy and prioritize cleaning examples from classes that are sensitive to noise and frequently mislabeled.
• Experiment with label smoothing, which assigns small nonzero probabilities to incorrect classes. This can help reduce the impact of label noise.
• Research adversarial training and data augmentation approaches to make your models more robust to noisy inputs.

There‘s also an important human element to dataset curation and model development. Establish clear labeling guidelines and train annotators carefully. Foster a culture of open communication and transparency around data quality issues. Encourage your teams to proactively surface and escalate examples of data pollution rather than letting them silently degrade model performance.

By promoting honesty and rigor in dataset development, we can avoid lying to our models (even unintentionally) and reap the benefits of cleaner data – more reliable and robust AI systems. The road to better performance isn‘t always paved with bigger datasets, but rather with higher quality ones.

As the stakes and scope of AI deployment grows, holding ourselves and our datasets to a high standard of truthfulness and integrity will become all the more critical. To avoid the pitfalls of "garbage in, garbage out", we must remain vigilant about the little lies, mistakes, and manipulations that can throw models off track. The truth, as they say, shall set you free – or at least make your models work a whole lot better.