Supercharge Your NLP Models with Text Data Augmentation using NLPAUG

Data augmentation has become a go-to technique to enhance model performance and generalization in machine learning. While extensively used in computer vision, it is less prevalent for NLP tasks. However, with the increasing adoption of deep learning in NLP, text data augmentation is gaining traction as an effective way to improve model robustness.

In this post, we‘ll dive into the world of text data augmentation using the powerful Python library NLPAUG. You‘ll learn various augmentation techniques and how to apply them effortlessly with NLPAUG to upskill your NLP models. Let‘s get started!

Why Text Data Augmentation Matters

Data augmentation involves creating new synthetic training examples by applying transformations to the original data. The intent is to expand the training dataset with new, diverse samples to improve the model‘s ability to generalize well on unseen data. Some key benefits of data augmentation are:

1. Enhancing model robustness and reducing overfitting, especially with smaller datasets
2. Boosting performance on data with noise, such as spelling errors, synonyms, or paraphrases
3. Overcoming class imbalance by oversampling minority classes

While data augmentation is extensively used in computer vision, it is still an emerging trend in NLP. Intuitively, techniques like cropping, rotation, or color jittering for images don‘t translate well to text. However, there are many techniques to augment textual data that preserve semantic meaning. Let‘s take a look.

Text Data Augmentation Techniques

Here are some popular techniques used for augmenting text data:

1. Synonym Replacement: Replacing words with their synonyms
   Original: The movie was very good
   Augmented: The film was very decent

2. Back Translation: Translating text to another language and back to the original
   Original: I went to the park yesterday
   Augmented: I visited the park yesterday

3. Random Insertion: Randomly inserting words in the sentence
   Original: The children loved the gifted toys
   Augmented: The cute children loved the gifted toys very much

4. Random Swap: Randomly swapping words in the sentence
   Original: The dog played fetch with the ball
   Augmented: The ball played fetch with the dog

5. Random Deletion: Randomly removing words from the sentence
   Original: My favorite sport is soccer since I love the excitement
   Augmented: My favorite sport is soccer since I love

As you can see, the augmented examples, while not perfect, preserve a good amount of the original meaning. The goal is to generate plausible variations of the text to teach models a more robust understanding beyond rote pattern matching.

Introducing NLPAUG

While you could code up these augmentations from scratch, the NLPAUG library provides a high-level, easy-to-use API for a wide variety of augmentation techniques. It greatly simplifies the process of augmenting your text data.

Some key features of NLPAUG are:

1. Simple, consistent API for applying augmentations
2. Support for character, word, and sentence level augmentations
3. Easy integration with popular NLP libraries like spaCy, nltk, gensim
4. Leveraging pre-trained models like word2vec, GloVe, BERT for contextual augmentation
5. Flow components to chain together multiple augmentation techniques

Here‘s how you can quickly get started with NLPAUG:

!pip install nlpaug

import nlpaug.augmenter.char as nac
import nlpaug.augmenter.word as naw

NLPAUG provides augmenters for character, word, and sentence/flow levels. Let‘s dive into each of them, with code samples.

Character-level Augmentation

Character-level augmentation involves modifying individual characters in the text. This can be useful to simulate noise like keyboard errors or OCR mistakes. NLPAUG provides intuitive APIs for it:

import nlpaug.augmenter.char as nac

text = "I love playing with my pet dog"

# Simulate keyboard errors
aug = nac.KeyboardAug()
augmented_text = aug.augment(text)
print("Original:")
print(text)
print("Augmented:")
print(augmented_text)

# Simulate OCR errors
aug = nac.OcrAug()
augmented_text = aug.augment(text)  
print("Original:")
print(text)
print("Augmented:")
print(augmented_text)

Output:

Original:
I love playing with my pet dog
Augmented:
I love pkaying with my pet dig

Original:
I love playing with my pet dog 
Augmented:
I 10ve p14y1ng with my pet d0g

The KeyboardAug simulates real-world keyboard errors by substituting characters close to each other on a standard QWERTY keyboard. The OcrAug mimics errors made by OCR systems, like incorrectly identifying ‘I‘ as ‘1‘.

Word-level Augmentation

Word-level augmentation involves inserting, modifying or replacing individual words or phrases. NLPAUG provides a variety of techniques for this:

import nlpaug.augmenter.word as naw

text = "The quick brown fox jumps over the lazy dog"

# Synonym replacement
aug = naw.SynonymAug(aug_src=‘wordnet‘)
augmented_text = aug.augment(text)
print("Original:")
print(text)
print("Augmented:")
print(augmented_text)

# Word embeddings substitution
aug = naw.WordEmbsAug(
    model_type=‘word2vec‘, model_path=‘GoogleNews-vectors-negative300.bin‘)
augmented_text = aug.augment(text)
print("Original:")
print(text)
print("Augmented:")
print(augmented_text)

# Contextual word embeddings (BERT)
aug = naw.ContextualWordEmbsAug(
    model_path=‘bert-base-uncased‘, action="insert")
augmented_text = aug.augment(text)
print("Original:")
print(text)
print("Augmented:")
print(augmented_text)

Output:

Original:
The quick brown fox jumps over the lazy dog
Augmented:
The quick brown fox leaps over the lazy dog

Original:
The quick brown fox jumps over the lazy dog
Augmented:
The speedy brown fox jumps over the lazy dog

Original:
The quick brown fox jumps over the lazy dog
Augmented:
The quick brown fox jumps over the lazy dog, which is sleeping.

The SynonymAug replaces words with WordNet synonyms. The WordEmbsAug substitutes words based on similarity in a pre-trained word2vec or GloVe vector space. This captures both synonyms and other strongly related words.

The ContextualWordEmbsAug is very powerful, as it leverages contextual embeddings from pre-trained language models like BERT. The augmented text is generated by a deep learning model that captures rich linguistic knowledge and long-range dependencies. This augmenter can insert or substitute words/phrases that are highly plausible in context.

Sentence-level & Flow Augmentation

NLPAUG also supports augmentation at the sentence level and allows you to chain together multiple augmenters via its Flow API. Some quick examples:

import nlpaug.augmenter.sentence as nas
import nlpaug.flow as naf

text = "What is your recommended book on Bayesian statistics?"

# Backtranslation 
aug = nas.BackTranslationAug()
augmented_text = aug.augment(text)
print("Original:")
print(text)
print("Augmented:")
print(augmented_text)

# Contextual word embeddings (BERT) for sentence
aug = nas.ContextualWordEmbsForSentenceAug(model_path=‘bert-base-uncased‘)
augmented_texts = aug.augment(text, n=3)
print("Original:")
print(text)
print("Augmented:")
print(augmented_texts)

# Sequential flow (multiple augmenters)
aug = naf.Sequential([
    nac.OcrAug(),
    naw.WordEmbsAug(model_type=‘word2vec‘, model_path=‘GoogleNews-vectors-negative300.bin‘, action="substitute")
])
augmented_text = aug.augment(text, n=3)
print("Original:")
print(text)
print("Augmented:")
print(augmented_text)

Output:

Original:
What is your recommended book on Bayesian statistics?
Augmented:
What is your recommended book on Bayesian statistics?

Original:
What is your recommended book on Bayesian statistics?
Augmented:
[‘What is your favorite book on Bayesian statistics?‘,
 ‘What is your recommended textbook on Bayesian statistics?‘,
 ‘What book do you recommend for learning Bayesian statistics?‘]

Original:
What is your recommended book on Bayesian statistics?  
Augmented:
[‘What is your suggested book on Bayesian statistics?‘,
 ‘What is your recommended textbook on Bayesian statistics?‘, 
 ‘What is your advised book on Bayesian statistics?‘]

The BackTranslationAug translates the text to another language and back, generating paraphrased examples. The ContextualWordEmbsForSentenceAug uses a pre-trained language model to generate sentence-level insertions that maintain coherence.

Finally, the Sequential flow showcases how you can combine character and word-level augmentations (or any arbitrary sequence) to generate highly diverse augmented examples.

Tuning the Augmentation

An important consideration when applying augmentation is tuning the degree of modification to suit your specific task and data. Too little augmentation may not add much value, while too much can distort the original meaning. NLPAUG provides several parameters to control aspects like:

1. The number or fraction of words to modify (e.g. aug_p, aug_min, aug_max)
2. The pool of words to sample from (e.g. top_k, top_p)
3. Constraints on length of insertions/substitutions (e.g. min_char, max_char)

Here‘s an example of tuning the degree of augmentation:

# Tune fraction of words to augment
aug = naw.WordEmbsAug(model_type=‘word2vec‘, model_path=‘GoogleNews-vectors-negative300.bin‘, 
    action="substitute", aug_p=0.1, top_k=100)  

This augmenter will substitute approximately 10% of words, sampling from the top 100 most similar words based on the word2vec embeddings. You should experiment with different settings to arrive at a degree of augmentation that provides a useful training signal for your models.

Best Practices & Alternatives

When leveraging text data augmentation for your NLP workflows, consider the following best practices:

1. Start with light augmentation and gradually increase intensity as needed
2. Tailor your augmentation strategy to the task – more conservative for sentiment/intent, more aggressive for paraphrasing/QA
3. Evaluate augmented examples manually to ensure meaning is preserved
4. Experiment with different augmenters and parameters
5. Monitor model performance on a held-out validation set

It‘s also good to be aware of other augmentation libraries. Some popular alternatives are:

1. TextAttack – Generates adversarial examples for NLP attacks/defenses
2. NLPAugment – Set of 11 augmentation techniques proposed in academic literature
3. TextAugment – Another library for word/sentence-level augmentation

Compared to these alternatives, some key advantages of NLPAUG are its extensive collection of augmenters, easy-to-use API, and ability to leverage state-of-the-art pre-trained models. It‘s a great all-in-one tool to have in your NLP toolbelt.

Conclusion

In this post, we explored the exciting field of text data augmentation and how to implement it with the awesome NLPAUG library. From character-level noise injection to contextual paraphrasing with BERT, NLPAUG puts a versatile set of augmentation techniques at your fingertips.

We walked through code examples for different augmenters, discussed how to tune the augmentation degree, and covered some best practices and alternatives. By leveraging NLPAUG to automatically generate high-quality augmented text, you can significantly improve your NLP models‘ robustness and performance.

So what are you waiting for? Go try it out on your datasets and see the results for yourself! A whole new (augmented) world awaits. Happy augmenting!