A Beginner‘s Guide to Natural Language Processing Using spaCy

Natural Language Processing, or NLP for short, is a branch of artificial intelligence that focuses on enabling computers to understand, interpret, and generate human language. NLP combines insights from computer science, linguistics, and machine learning to build intelligent systems that can process and analyze large amounts of unstructured text data.

Why NLP Matters

In the era of Big Data, an enormous amount of information is generated in the form of unstructured text – emails, social media posts, news articles, scientific papers, medical records, and more. According to IBM, unstructured text accounts for over 80% of all data generated globally. NLP provides the tools and techniques to automatically process and extract valuable insights from this vast trove of textual data.

NLP has a wide range of applications across industries:

• In healthcare, NLP is used to analyze electronic health records, identify patients at risk of certain conditions, and extract key information from clinical notes.
• In finance, NLP powers sentiment analysis of news and social media to predict stock price movements, and enables chatbots to handle customer inquiries.
• In e-commerce, NLP is used for product categorization, sentiment analysis of reviews, and question-answering systems.
• In education, NLP is used to grade student essays, provide personalized feedback, and develop intelligent tutoring systems.

The global NLP market size was valued at USD 10.72 billion in 2020 and is projected to reach USD 48.46 billion by 2026, growing at a CAGR of 26.84% during the forecast period (2021-2026) (source: Mordor Intelligence).

Introducing spaCy

spaCy is a free, open-source library for advanced Natural Language Processing in Python and Cython. It was created by Matthew Honnibal and Ines Montani, the founders of Explosion AI. Since its release in 2015, spaCy has become one of the most popular and fastest-growing NLP libraries.

Some of spaCy‘s key features include:

• Non-destructive tokenization
• Named Entity Recognition (NER)
• Part-of-speech (POS) tagging
• Dependency parsing
• Sentence segmentation
• Integrated word vectors
• Convenient string-to-hash mapping
• Export to NumPy data arrays
• GIL-free multi-threading
• Concise API with consistent interfaces

spaCy is designed with a focus on developer experience and practical usage. Its API is straightforward and well-documented, and it provides a simple and consistent interface to common NLP tasks. spaCy also places a strong emphasis on performance, with speed benchmarks showing it to be the fastest NLP library in many tasks.

Source: spaCy benchmarks

Under the hood, spaCy is powered by statistical models trained on large corpora of text data. spaCy provides a variety of pre-trained models for different languages and domains, which can be easily loaded and used for prediction. It also provides tools and workflows for training your own custom models on your specific data.

Installing spaCy

Let‘s dive into using spaCy for NLP tasks. First, make sure you have Python installed (spaCy requires Python 3.6+). Then, you can install spaCy using pip:

pip install -U spacy

Next, you‘ll need to download a pre-trained model. For this guide, we‘ll use the en_core_web_sm model, which is a small English model trained on web text:

python -m spacy download en_core_web_sm

spaCy‘s Core Data Structures

Before we explore spaCy‘s functionality, let‘s understand its core data structures:

• Doc: A container for accessing linguistic annotations.
• Token: An individual token — i.e. a word, punctuation symbol, whitespace, etc.
• Span: A slice from a Doc object, consisting of one or more tokens.
• Lexeme: An entry in the vocabulary, containing a unique string and its associated linguistic attributes.

When you process a text with spaCy, it is tokenized and converted into a Doc object. The Doc is then processed by a series of components in a pipeline, each of which adds a specific type of annotation. Here‘s an illustration of spaCy‘s pipeline architecture:

Source: spaCy documentation

Linguistic Annotations with spaCy

Now let‘s see how to use spaCy to perform common NLP tasks. We‘ll use the following text as a running example:

text = "David Bowie was an English singer-songwriter and actor. He was a leading figure in the music industry and is regarded as one of the most influential musicians of the 20th century."

Tokenization

Tokenization is the process of splitting a text into individual tokens, which roughly correspond to "words". spaCy provides a fast and robust tokenizer that handles a variety of cases:

import spacy

nlp = spacy.load("en_core_web_sm")
doc = nlp(text)

for token in doc:
    print(token.text, token.pos_, token.dep_)

This will print each token, its part-of-speech tag, and its dependency label:

David PROPN compound
Bowie PROPN nsubj
was AUX ROOT
an DET det
English ADJ amod
singer NOUN compound
- PUNCT punct
songwriter NOUN conj
and CCONJ cc
actor NOUN conj
...

Named Entity Recognition

Named Entity Recognition (NER) is the task of identifying and classifying named entities in text into pre-defined categories such as person names, organizations, locations, etc.

for ent in doc.ents:
    print(ent.text, ent.label_)

Output:

David Bowie PERSON
English NORP
20th century DATE

spaCy‘s NER model is trained to recognize a wide range of named entities out-of-the-box, and can also be trained on custom categories using spaCy‘s training APIs.

Part-of-speech Tagging

Part-of-speech (POS) tagging is the process of marking up a word in a text as corresponding to a particular part of speech, based on both its definition and its context. spaCy provides a fast and accurate POS tagger:

for token in doc:
    print(token.text, token.pos_)

Output:

David PROPN
Bowie PROPN
was AUX
an DET
English ADJ
singer NOUN
...

spaCy uses the OntoNotes 5 version of the Penn Treebank tag set for its POS tags. This is a fine-grained tag set that captures detailed grammatical distinctions.

Dependency Parsing

Dependency parsing is the task of analyzing the grammatical structure of a sentence and establishing relationships between "head" words and words which modify those heads. spaCy‘s dependency parser is based on a non-projective transition-based parser:

for token in doc:
    print(token.text, token.dep_, token.head.text)

Output:

David compound Bowie
Bowie nsubj was
was ROOT was
an det figure
English amod singer
singer compound figure
...

The .dep_ attribute gives the dependency label, and .head gives the syntactic head token. These attributes together form a dependency tree that captures the grammatical structure of the sentence.

Word Vectors and Similarity

spaCy‘s medium and large models include pre-trained word vectors, also known as word embeddings. These are dense, continuous representations of words that capture semantic and syntactic similarities. You can access a token‘s vector via the .vector attribute:

king = nlp("king").vector
queen = nlp("queen").vector

print(king.dot(queen)) 

This will print the cosine similarity between the vectors for "king" and "queen", which should be quite high due to their semantic relatedness.

spaCy also provides a .similarity() method that computes the similarity between two tokens, spans, or documents:

doc1 = nlp("The movie was great!")
doc2 = nlp("The film was excellent!")

print(doc1.similarity(doc2))

This will print a similarity score between 0 and 1, based on the similarity of the vectors of the tokens in each document.

Building NLP Applications with spaCy

Now that we‘ve covered the basics of spaCy‘s functionality, let‘s look at how to use it to build real-world NLP applications.

Sentiment Analysis

Sentiment analysis is the task of classifying the polarity of a given text – whether the expressed opinion is positive, negative, or neutral. While spaCy doesn‘t include a built-in sentiment analysis model, we can build one using spaCy‘s features and a machine learning classifier.

Here‘s a simplified example using spaCy and scikit-learn:

import spacy
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

nlp = spacy.load("en_core_web_lg")

# Prepare data
texts = [
    "This movie was great!",
    "The film was terrible.",
    "The movie was okay, but not amazing.",
    ...
]
labels = [1, 0, 0, ...]  # 1 for positive, 0 for negative

# Get document vectors
doc_vectors = [nlp(text).vector for text in texts]

# Train classifier
X_train, X_test, y_train, y_test = train_test_split(doc_vectors, labels, test_size=0.2)
clf = LogisticRegression(random_state=0).fit(X_train, y_train)

# Evaluate
print(clf.score(X_test, y_test))

# Predict
print(clf.predict([nlp("This movie was fantastic!").vector]))

This trains a logistic regression model on the document vectors to predict the sentiment label. We can use this model to predict the sentiment of new, unseen texts.

Knowledge Graph Creation

A knowledge graph is a graph-structured knowledge base that integrates information from various sources. We can use spaCy to extract entities and relationships from text to build a knowledge graph.

Here‘s a simple example using Wikipedia data:

import spacy
from spacy import displacy

nlp = spacy.load("en_core_web_lg")

text = "Apple Inc. is an American multinational technology company headquartered in Cupertino, California. It was founded by Steve Jobs, Steve Wozniak, and Ronald Wayne in April 1976."

doc = nlp(text)

entities = [(ent.text, ent.label_) for ent in doc.ents]
print("Entities:", entities)

# Visualize
displacy.serve(doc, style="ent")

# Extract subject-verb-object triples
triples = []
for ent in doc.ents:
    if ent.dep_ in ("nsubj", "nsubjpass"):
        verb = ent.head
        for child in verb.children:
            if child.dep_ == "dobj":
                triples.append((ent.text, verb.lemma_, child.text))

print("Triples:", triples)

This extracts entities and subject-verb-object triples from the text, which can be used as the nodes and edges of a knowledge graph.

Challenges and Frontiers in NLP

Despite the impressive progress in NLP over the past few years, there are still significant challenges to overcome:

• Ambiguity: Human language is full of ambiguity, which can be lexical (a word having multiple meanings), syntactic (a sentence having multiple parse trees), or semantic (a sentence having multiple interpretations). Resolving ambiguity often requires understanding the broader context and world knowledge.

• Sarcasm and irony: Detecting sarcasm and irony is a difficult task for NLP systems, as it often requires understanding tone, context, and even cultural knowledge.

• Analogical reasoning: The ability to understand and generate analogies is a hallmark of human intelligence. NLP systems struggle with tasks that require analogical reasoning, like understanding metaphors or solving word analogies.

• Common sense reasoning: Much of human language understanding relies on a vast amount of implicit common sense knowledge about the world. Endowing NLP systems with this kind of knowledge and reasoning capability is an open challenge.

At the frontier of NLP research are efforts to build systems that can truly understand and engage with human language, rather than just perform narrow tasks. Some exciting directions include:

• Multimodal NLP: Integrating NLP with computer vision and speech processing to understand and generate multimodal content, such as images with captions or videos with subtitles.

• Multilingual and cross-lingual NLP: Building NLP models that can handle multiple languages and transfer knowledge between languages.

• Explainable AI for NLP: Developing methods to make the decisions and predictions of NLP models more interpretable and explainable to humans.

• Ethical and responsible NLP: Addressing issues of bias, fairness, transparency, and accountability in NLP models and applications.

Learning More

This guide has provided a high-level overview of NLP and how to get started with spaCy, but there‘s so much more to learn. Here are some excellent resources to continue your NLP journey:

• Natural Language Processing with Python (aka the "NLTK Book"): A comprehensive introduction to NLP concepts and techniques using Python and the NLTK library.

• Speech and Language Processing (3rd ed. draft): A textbook by Dan Jurafsky and James H. Martin covering a wide range of topics in NLP, from language modeling to semantic interpretation.

• Stanford CS224n: Natural Language Processing with Deep Learning: A popular course on NLP using deep learning methods. Lecture videos and materials are available online.

• ACL Anthology: A digital archive of research papers in computational linguistics and NLP. An excellent resource for staying up-to-date with the latest research.

• NLP News: A weekly newsletter covering the latest happenings in NLP, curated by Sebastian Ruder.

NLP is a vast and rapidly evolving field, with new techniques and applications emerging all the time. By mastering the fundamentals and staying curious and engaged with the community, you‘ll be well-equipped to apply NLP to solve real-world problems and contribute to the exciting future of this field.