Unleashing the Power of Custom Named Entity Recognition with spaCy v3

Named entity recognition (NER) is a crucial natural language processing (NLP) task that automatically detects and classifies named entities such as persons, organizations, locations, dates, quantities, and more within unstructured text. NER powers a wide range of applications including information extraction, text summarization, question answering, content recommendation, and knowledge base construction.

While pre-trained NER models can handle common entity types, many domains require identifying custom entity categories such as product names, biomedical terms, or legal clauses. That‘s where custom NER comes in – allowing you to train your own NER model on labeled examples of domain-specific entities.

spaCy is the leading open-source library for industrial-strength NLP in Python. As of version 3.0 released in 2020, spaCy makes training custom NER models more accessible and powerful than ever before. Built on the latest deep learning architectures, spaCy v3 delivers blazing fast training and inference, straightforward configuration, and seamless integration with the rest of the spaCy ecosystem for end-to-end NLP pipelines.

In this guide, we‘ll walk through the complete workflow of building a custom NER model using spaCy v3. Whether you‘re a machine learning practitioner, data scientist, or software engineer, you‘ll gain the knowledge and code to tackle entity extraction in your domain of choice. Let‘s dive in!

Preparing Your NER Training Data

The first step is generating training data in the format expected by spaCy v3. This format consists of text samples annotated with entity spans and their corresponding labels. While you can hand-label data using an annotation tool like Prodigy, we‘ll assume you already have pre-annotated data.

A common format is a list of tuples, where each tuple contains a text sample string and a dictionary mapping entity spans to labels. Here‘s an example for a custom "FOOD" entity:

train_data = [
    ("I love eating strawberries in the summer.", {"entities": [(15, 27, "FOOD")]}),
    ("Avocado toast is my favorite breakfast.", {"entities": [(0, 7, "FOOD")]}),
    ...
]

To convert this to spaCy‘s binary format for efficient training, we‘ll create a DocBin to store the training corpus:

from spacy.tokens import DocBin

db = DocBin()
for text, annotations in train_data:
    doc = nlp.make_doc(text)
    ents = []
    for start, end, label in annotations["entities"]:
        span = doc.char_span(start, end, label=label)
        ents.append(span)
    doc.ents = ents
    db.add(doc)
db.to_disk("./train.spacy")

This converts the annotated examples to spaCy Doc objects which are serialized into a compressed binary file for faster loading during training.

Make sure to set aside a portion of your annotated data as a validation set to evaluate your trained model‘s performance. We‘ll store the validation examples in a separate ./valid.spacy file.

Configuring Your Custom NER Model

Next, we need to configure the architecture and hyperparameters of our custom NER model. spaCy v3 provides a convenient interactive widget on their website to generate a configuration file template based on your requirements.

Simply visit the quickstart guide, choose "Named Entity Recognition" as the component, select your desired hardware and optimization settings, then download the auto-generated config.cfg file.

The config file uses spaCy‘s declarative model definition language to specify the model architecture, loss function, optimizer, training schedule, and more. Here‘s a condensed example:

[paths]
train = "corpus/train.spacy"
dev = "corpus/valid.spacy"

[training]
seed = ${vars.seed}
gpu_allocator = ${vars.gpu_allocator}
dropout = 0.1
accumulate_gradient = 1

[training.batcher]
@batchers = "spacy.batch_by_words.v1"
discard_oversize = false
tolerance = 0.2

[training.optimizer]
@optimizers = "Adam.v1"

[training.optimizer.learn_rate]
@schedules = "warmup_linear.v1"
initial_rate = 0.01

[model]
@architectures = "spacy.TransitionBasedParser.v2"

[model.tok2vec]
@architectures = "spacy.Tok2VecListener.v1"

[model.tok2vec.embed]
@architectures = "spacy.MultiHashEmbed.v1"

[model.ner]
@architectures = "spacy.TransitionBasedParser.v2"

This config uses a transition-based approach to NER, a multi-hash embedding table for tokens, and a linear warmup schedule for the learning rate. Feel free to tweak the settings based on your data size and domain complexity.

Training Your Custom NER Model

With our training data and configuration ready, we can now train our custom NER model! From the command line, simply run:

python -m spacy train config.cfg --output ./output

spaCy will load the training and validation datasets according to the [paths] block in the config file. The --output directory is where checkpoints and the final trained model will be saved.

During training, spaCy will print metrics for each epoch such as:

============================ Training pipeline ============================
ℹ Pipeline: [‘tok2vec‘, ‘ner‘]
ℹ Initial learn rate: 0.001
E    #       LOSS TOK2VEC  LOSS NER  ENTS_F  ENTS_P  ENTS_R  SCORE 
---  ------  ------------  --------  ------  ------  ------  ------
  1       0          0.00     26.50    0.21    0.67    0.12    0.00
  1     200         33.08   3975.78   38.10   45.45   32.65    0.04
  1     400         69.83   2963.07   70.97   77.27   65.65    0.11
  1     600         60.63   3221.49   78.95   77.36   80.61    0.17
  2       0         50.14   2238.06   91.67   88.00   95.65    0.37
  2     200         34.95   1299.42   93.24   92.06   94.44    0.49

The score represents the overall model accuracy on the validation set, which ideally should increase as training progresses. spaCy will also autosave the best performing model weights based on this metric.

By default, spaCy will train for a specified number of steps defined in the config file. However, you can manually stop training early by hitting Ctrl+C once performance plateaus.

Evaluating Your Custom NER Model

Once training finishes, it‘s time to see how well our custom NER model actually performs on unseen text. We‘ll load the trained model from disk and use it to make predictions:

import spacy

nlp = spacy.load("./output/model-best")
doc = nlp("For breakfast I had avocado toast and coffee.")
print(doc.ents)

(avocado toast, coffee)

spaCy returns a tuple of named entity spans detected by our custom model. We can see it successfully identified the two food items in the sentence.

To get a more quantitative evaluation, we can run the model on a labeled test set and compute metrics like precision, recall, and F1 score:

scorer = nlp.evaluate(test_data)
print(scorer.scores)

{
    ‘ents_f‘: 0.9315668202764977,
    ‘ents_p‘: 0.9466666666666667, 
    ‘ents_r‘: 0.92
    ‘token_acc‘: 0.9966027874564459,
    ‘textcat_score‘: 0.0,
    ‘speed‘: 8904.6171875
}

The ents_ scores reflect the model‘s performance on extracting named entities, while token_acc measures the accuracy of the underlying tokenization and speed reports the tokens processed per second.

For a more intuitive visualization of errors, we can use the displacy renderer:

from spacy import displacy

colors = {"FOOD": "yellow"}
options = {"ents": ["FOOD"], "colors": colors}
displacy.render(doc, style="ent", jupyter=True, options=options)

Displacy highlights the extracted entities in context which helps identify patterns of false positives and false negatives to address.

Improving Your Custom NER Model

Once you‘ve assessed the strengths and weaknesses of your initial model, there are a few key ways to boost its performance:

1. Add more training data, especially examples of entity edge cases and ambiguities

2. Tune the model hyperparameters like the learning rate, batch size, and dropout

3. Experiment with different word embedding configurations like switching to transformers

4. Stack NER with other components like a POS tagger or dependency parser for richer linguistic context

5. Incorporate domain-specific gazetteers and regex patterns to complement ML predictions

The spaCy documentation provides excellent guidance on customizing and modularizing your pipeline for your particular domain and deployment needs.

Real-World Applications of Custom NER

Custom NER models enable powerful information extraction across diverse domains such as:

• Extracting product attributes like brands, materials, and sizes from ecommerce listings
• Identifying medical terms like drugs, diseases, and procedures in clinical notes
• Detecting legal entities like contracts, clauses, and parties in legal documents
• Recognizing unique person and place names in historical archives
• Pulling out key metrics and KPIs from financial reports and earnings call transcripts

By training spaCy v3 on your own labeled data, you can unlock valuable named entities to drive automation, business intelligence, and user-facing features.

Conclusion and Future Directions

In this guide, we‘ve walked through the entire lifecycle of building a custom named entity recognition model using the state-of-the-art spaCy v3 library. From data preparation, to model configuration, to training and evaluation, you now have the knowledge and code to create NER models optimized for your domain.

As you embark on your custom NER journey, remember that effective entity extraction combines machine learning with domain expertise and heuristics. Don‘t hesitate to experiment, iterate, and layer different approaches to drive the best outcomes.

Looking ahead, the field of NER continues to evolve with exciting developments like:

• Adapting pre-trained language models like BERT for finer-grained entity typing
• Injecting entity-centric knowledge via knowledge graphs and entity linking
• Cross-lingual and multi-task NER to handle multilingual documents and leverage auxiliary objectives

The spaCy team remains at the forefront of NLP innovation, with a active community and frequent releases to incorporate state-of-the-art techniques.

To stay up to date with the latest in NER and spaCy v3, I recommend the following resources:

• spaCy v3 documentation
• Explosion AI blog
• spaCy GitHub repository

You can also connect with me on LinkedIn or Twitter to discuss all things NLP and swap NER war stories.

I‘d love to hear how you‘re applying custom NER in your work – feel free to share your experiences and insights in the comments below!

Until next time, happy entity extraction!