Knowledge Graph in Machine Learning (Explained)

How to Build Knowledge Graphs from Text Using spaCy
Knowledge graphs are a powerful way to represent information by modeling entities and the relationships between them. They provide computers with a way to contextualize data points and understand how different concepts are semantically related. Some famous examples of knowledge graphs include the Google Knowledge Graph, which enhances the search engine‘s results with information gathered from a variety of sources, and IBM‘s Watson, which used knowledge graphs to beat human champions on Jeopardy!

Knowledge graphs have many useful applications in natural language processing (NLP) and artificial intelligence, from powering sophisticated question-answering systems to providing intelligent recommendations. In this article, we‘ll take a deep dive into how you can automatically build your own knowledge graph from raw text data using the popular open-source NLP library, spaCy. Whether you‘re new to NLP or an experienced practitioner, this guide will walk you through the key steps in the knowledge graph construction pipeline.

Overview of Knowledge Graph Construction from Text
Unstructured text is messy and cannot be directly used to build a knowledge graph. We first need to process the raw text data to identify the relevant entities and understand the natural language descriptions of relationships between those entities. This is where powerful NLP techniques come into play. Here‘s a high-level overview of the knowledge graph construction process:

1. Sentence Segmentation: The first step is to break down a text document into individual sentences, as most relations are expressed within the scope of a sentence. This allows us to process each sentence independently to extract entities and relationships.

2. Named Entity Recognition (NER): Next, we need to identify mentions of named entities in each sentence. These could be people, places, organizations, and so on, which will form the nodes in our knowledge graph. spaCy provides a pre-trained NER model that can recognize a wide range of named entities out-of-the-box.

3. Dependency Parsing: To extract relationships between entities, we need to understand the grammatical structure of the sentence. Dependency parsing analyzes the syntactic structure of a sentence to determine the relationships between words. spaCy‘s dependency parser is able to generate a tree representing the grammatical structure of a sentence.

4. Relation Extraction: With the named entities and dependency parse tree for each sentence, we‘re now ready to extract semantic relationships between the entities. We can define patterns to identify common types of relations based on the dependency labels and entity types. More advanced techniques like Open Information Extraction can automatically extract relation phrases from text.

5. Graph Construction: The final step is to construct the knowledge graph based on the extracted entities and relations. Each unique entity becomes a node in the graph, and each relation becomes an edge connecting two nodes. The result is a graph data structure that captures the key concepts and semantic relationships mentioned in the input text.

Now that you have an overview of the process, let‘s dive into the implementation details and see how to use spaCy to build a knowledge graph from scratch!

Step 1: Sentence Segmentation
The first step is to split a text document into individual sentences. This is important because relationships between entities are typically expressed within the boundaries of a sentence. spaCy provides a fast and accurate sentence segmenter that we can use out of the box.

Assuming you have a text document loaded into a string variable called text, here‘s how you can use spaCy to perform sentence segmentation:

import spacy

nlp = spacy.load("en_core_web_sm")
doc = nlp(text)

sentences = list(doc.sents)

The spacy.load() function is used to load a pre-trained English language model. We then process the input text using the nlp object, which creates a spaCy Doc object. The doc.sents property returns a generator of sentence spans, which we convert to a list to get a list of segmented sentences.

Step 2: Named Entity Recognition
The next step is to identify mentions of named entities within each sentence. These entities could represent people, places, organizations, products, etc. and will become the nodes in our knowledge graph. spaCy provides a state-of-the-art named entity recognizer that can identify a wide range of entity types with high accuracy.

To perform named entity recognition on the segmented sentences, we can simply iterate over each sentence and access the ents property of the sentence span:

entities = []
for sent in sentences:
    entities.extend([(e.text, e.label_) for e in sent.ents])

This code snippet extracts the text and label of each named entity recognized within each sentence and appends them to the entities list. The label_ attribute contains the predicted entity type, such as "PERSON", "ORG", "GPE", etc.

Step 3: Dependency Parsing
To extract relationships between entities, we need to understand the grammatical structure of each sentence. This is where dependency parsing comes in. Dependency parsing analyzes the syntactic structure of a sentence and determines the relationships between words based on a dependency grammar.

spaCy provides a fast and accurate dependency parser that we can use to generate a dependency parse tree for each sentence. Here‘s how to parse the dependencies of the segmented sentences:

for sent in sentences:
    print(sent.text)
    for token in sent:
        print(f"{token.text}({token.pos_}) --> {token.dep_} --> {token.head.text}({token.head.pos_})")
    print()

This code loops through each sentence, prints the sentence text, and then iterates over each token in the sentence. For each token, it prints the token text, part-of-speech tag (POS), dependency label, and the head token it depends on. This allows us to visualize the dependency parse tree for each sentence.

Step 4: Relation Extraction
With the named entities and dependency parse information, we can now extract semantic relationships between entities. There are different approaches to relation extraction, ranging from rule-based methods to machine learning models. For this example, we‘ll use a simple rule-based approach based on dependency patterns.

The idea is to define patterns that capture common ways relationships are expressed based on the dependency parse tree. For example, a pattern for extracting relations could be: [ENTITY] <nsubj< [ROOT] >dobj> [ENTITY]. This pattern looks for a subject entity connected to a root verb, which in turn has a direct object entity.

Here‘s a function that implements this pattern matching logic to extract subject-verb-object triples:

def extract_relations(sent):
    relations = []
    for ent1 in sent.ents:
        for ent2 in sent.ents:
            if ent1 != ent2:
                if ent1.start < ent2.start:
                    for token in sent:
                        if token.dep_ == "ROOT" and token.pos_ == "VERB":
                            subject = None
                            object = None
                            for child in token.children:
                                if child.dep_ == "nsubj" and child.subtree.contains(ent1):
                                    subject = ent1
                                if child.dep_ == "dobj" and child.subtree.contains(ent2):
                                    object = ent2
                            if subject and object:
                                relations.append((subject, token, object))
    return relations

This function takes a sentence span as input and returns a list of extracted relation triples in the form of (subject, relation, object). It loops through each pair of entities in the sentence and checks if they are connected by a ROOT verb in a subject-verb-object pattern. If a matching pattern is found, the relation triple is appended to the relations list.

We can extract relations from all sentences like this:

all_relations = []
for sent in sentences:
    relations = extract_relations(sent)
    all_relations.extend(relations)

The all_relations list will contain the extracted relation triples from all sentences in the input text.

Step 5: Graph Construction
The final step is to construct the actual knowledge graph from the extracted entities and relations. We‘ll represent the graph using the NetworkX library, where each unique entity becomes a node and each relation becomes an edge connecting two nodes.

First, let‘s install NetworkX and import it:

!pip install networkx
import networkx as nx

Now, we can create a directed graph and add the entities and relations as nodes and edges:

G = nx.DiGraph()

for entity in set(entities):
    G.add_node(entity[0], type=entity[1])

for relation in all_relations:
    G.add_edge(relation[0].text, relation[2].text, relation=relation[1].lemma_)

This code first creates a directed graph object using nx.DiGraph(). It then iterates over the unique entities and adds them as nodes to the graph. The entity text becomes the node label, and the entity type is stored as a node attribute.

Next, it loops through each extracted relation triple and adds an edge between the subject and object entities. The relation verb becomes the edge label, stored as an edge attribute.

We can visualize the constructed knowledge graph using NetworkX‘s drawing capabilities:

import matplotlib.pyplot as plt

pos = nx.spring_layout(G)
nx.draw(G, pos, with_labels=True, node_size=1000, font_size=12, font_weight=‘bold‘, node_color=‘lightblue‘, edge_color=‘gray‘, linewidths=2)
labels = nx.get_edge_attributes(G, ‘relation‘)
nx.draw_networkx_edge_labels(G, pos, edge_labels=labels)

plt.axis(‘off‘)
plt.show()

This code uses NetworkX‘s spring_layout to compute node positions for a visually appealing layout. It then draws the graph nodes and edges with customized styles. Edge labels are drawn separately using the draw_networkx_edge_labels function. Finally, the graph is displayed using matplotlib.

And there you have it! You‘ve successfully built a knowledge graph from raw text using spaCy and visualized it using NetworkX. The resulting graph captures the key entities and their relationships mentioned in the text, providing a structured representation of the knowledge.

Challenges and Future Directions
Building knowledge graphs from text is a complex task, and there are several challenges to consider:

1. Coreference Resolution: Identifying and resolving pronouns and other coreferences is crucial for extracting accurate relations. Without proper coreference resolution, the knowledge graph may miss important connections between entities.

2. Handling Compound Entities: Extracting multi-word entities and their relationships can be challenging. More advanced techniques like named entity linking and co-reference resolution may be necessary to accurately identify and link compound entities.

3. Filtering Insignificant Relations: Not all extracted relations are meaningful or relevant. Filtering out insignificant or overly specific relations is important to maintain the quality and usability of the knowledge graph.

4. Scalability: Building knowledge graphs from large volumes of text data can be computationally expensive. Efficient algorithms and distributed processing techniques may be necessary to scale the knowledge graph construction process.

Despite these challenges, knowledge graphs remain a powerful tool for representing and reasoning over complex information. As NLP techniques continue to advance, we can expect more sophisticated and accurate methods for constructing knowledge graphs from unstructured text.

Conclusion
In this article, we explored how to build a knowledge graph from text data using the spaCy library. We walked through the key steps involved, including sentence segmentation, named entity recognition, dependency parsing, relation extraction, and graph construction. By leveraging spaCy‘s powerful NLP capabilities, we were able to extract meaningful entities and relationships from raw text and represent them in a structured graph format.

Knowledge graphs have numerous applications in various domains, such as question answering, recommendation systems, information retrieval, and knowledge management. By providing a semantic representation of knowledge, they enable more intelligent and context-aware processing of textual data.

As you continue your journey in natural language processing and knowledge graph construction, I encourage you to explore more advanced techniques and experiment with different approaches. Leveraging machine learning models for relation extraction, incorporating external knowledge bases, and scaling up to larger datasets are just a few directions to consider.

Remember, building knowledge graphs is an iterative process that requires continuous refinement and domain expertise. The more you work with text data and understand the intricacies of language, the better you‘ll become at constructing meaningful and effective knowledge graphs.

I hope this guide has provided you with a solid foundation for building knowledge graphs from text using spaCy. Feel free to reach out if you have any questions or want to share your own experiences working with knowledge graphs. Happy graphing!