Building an Automatic Text Summarization Web App with SBERT and Flask: A Practical Guide

ByJordan Brown

The amount of textual data available on the internet and in enterprise databases is staggering and growing exponentially. According to IDC, the volume of digital data globally is projected to grow to 175 zettabytes by 2025, with much of that being unstructured text such as webpages, articles, social media posts, and business documents.

With this deluge of text, there is an urgent need for automated tools that can help people quickly make sense of all this information and extract actionable insights. One of the most promising NLP techniques for this is automatic text summarization – the process of distilling the key ideas and information from a longer text into a concise summary.

Some eye-opening statistics that highlight the need for text summarization:

  • The average person consumes 34 gigabytes of content and 100,000 words of information in a single day (Forbes)
  • The average attention span has shrunk by 25% in a few years, to around 8 seconds (Time)
  • 81% of people only skim the content they read online (Adobe)

Clearly, being able to quickly get the gist of a long article, report, or webpage is not a luxury but a necessity in our fast-paced, information-saturated world.

While summarizing text is a trivial task for humans, it‘s very challenging for machines. It requires truly understanding the meaning and importance of different passages of text, and being able to distill and paraphrase the essential points in coherent language. Traditional approaches like word frequency counting or TextRank often produce summaries that are extractive (verbatim passages of the original text) rather than abstractive, and lack semantic understanding.

In recent years, deep learning-based language models like BERT that are pre-trained on huge corpora have revolutionized the field of NLP. SBERT (Sentence-BERT) is a powerful extension of BERT that gives high quality sentence embeddings optimized for tasks like text similarity comparison. In this article, we‘ll see how to leverage SBERT to build a practical web app for extractive text summarization that achieves state-of-the-art results.

How SBERT Works for Extractive Summarization

SBERT is a modification of the pretrained BERT model that adds a pooling operation to the output of BERT, to derive a semantically meaningful fixed sized sentence embedding. Specifically, SBERT inserts either a mean or max pooling layer on top of the BERT outputs for each token in the input text. This generates a single 768 dimensional vector that represents the entire sentence or paragraph.

SBERT architecture diagram
Architecture of SBERT (Image source: SBERT paper)

These sentence embeddings generated by SBERT have some powerful properties. Sentences that express similar meanings are mapped close to each other in the embedding space, while sentences with dissimilar meanings are mapped far apart. The sentence embeddings can therefore be compared using cosine similarity to quantify the semantic similarity between sentences.

For extractive text summarization, the key idea is to find the sentences in a text that are most semantically similar to the overall meaning of the entire text. These are the most central and representative sentences that capture the main topics and ideas. The extractive summarization algorithm using SBERT roughly works like:

  1. Split the input text into individual sentences
  2. Obtain the SBERT embedding for each sentence
  3. Take the average of all sentence embeddings to get the overall document embedding
  4. Rank each sentence by cosine similarity to the document embedding
  5. Return the top N highest ranked sentences as the summary

This simple approach works surprisingly well for identifying the key sentences and assembling them into a useful summary, as we‘ll see shortly. More sophisticated algorithms can further improve the results by ensuring non-redundancy between the selected sentences and optimizing for coherence.

Some advantages of using SBERT for extractive summarization compared to other methods:

  • Pre-training on a huge amount of text data captures rich linguistic knowledge
  • The sentence embeddings encode semantic meaning, not just surface word frequencies
  • Very computationally efficient compared to comparing sentence pairs with vanilla BERT
  • Minimal training/fine-tuning needed, works well out of the box on many domains
  • Extractive summaries are guaranteed to be grammatical and faithful to original

Step-by-Step: Implementing the SBERT Summarization App

Now let‘s walk through building a web app that uses SBERT to generate extractive summaries of articles or reports. We‘ll use Python and Flask for the backend, and HTML/CSS for a simple frontend interface.

Prerequisites

Make sure you have Python 3.6+ installed, and install the required dependencies:

pip install Flask sentence-transformers bert-extractive-summarizer

Step 1: Set up the Flask app

Create a new Python file called app.py with the following code to initialize a Flask app:

from flask import Flask, request, render_template
from summarizer import Summarizer

app = Flask(__name__)
model = Summarizer()

@app.route(‘/‘)
def index():
    return render_template(‘index.html‘)

if __name__ == ‘__main__‘:
    app.run(debug=True)

This sets up a Flask server with a single route for the homepage, which will display the HTML form for inputting text to summarize. We load the default SBERT model via the bert-extractive-summarizer library.

Step 2: Define the summarization route

Next, add a /summarize route that handles the form submission and generates the summary:

@app.route(‘/summarize‘, methods=[‘POST‘])
def summarize():
    text = request.form[‘text‘]
    summary = model(text, num_sentences=3)
    return render_template(‘summary.html‘, text=text, summary=summary)

This takes the text input from the form, passes it to the model to extract a 3 sentence summary, and renders the summary.html template to display the output.

Step 3: Create the HTML templates

In a templates/ subdirectory, create two HTML files:

index.html



<form action="/summarize" method="post">
  <textarea name="text" rows="20" cols="100"></textarea>
  <br>
  <input type="submit" value="Summarize">
</form>

summary.html

<h2>Original Text</h2>
<p>{{ text }}</p>

<h2>Summary</h2>
<p>{{ summary }}</p>

<a href="/">Summarize another text</a>

Step 4: Run the app

Start the Flask server by running:

python app.py

Open up http://localhost:5000 in your web browser, and you should see the text input box. Paste an article or long text into the box, click "Summarize", and voila – you‘ll get a concise summary of the key points!

Seeing SBERT Summarization in Action

To get a sense of the quality of summaries generated by this SBERT-based approach, here are a few examples on different types of text.

News Article

Original text: A 500 word article about the impact of COVID-19 on the airline industry (link)

Extracted summary:

The Covid-19 pandemic has caused the worst financial crisis in the history of the airline industry. US airlines are slashing their flights by 70% to 90% in April and May. The airlines are currently losing about $350 million to $400 million a day as expenses like payroll, rent and aircraft maintenance far exceed the money they‘re bringing in.

Scientific Paper

Original text: The abstract and introduction from a paper on using BERT for text summarization (link)

Extracted summary:

In this paper, we study the impact of different choices when using BERT for extractive summarization. Extractive summarization aims at selecting important sentences from the document as the summary. The most recent work on extractive summarization using BERT only feeds the sentence representation into a classifier to obtain the probability of whether a sentence should be selected, which we argue is not enough. We find the improvement is mainly from better semantic representations of the sentences instead of sophisticated graph neural networks.

Financial Report

Original text: An excerpt from Berkshire Hathaway‘s 2019 annual report (link)

Extracted summary:

Charlie and I do not view the $248 billion detailed above as a collection of stock market wagers – dalliances to be terminated because of downgrades by "the Street," expected Federal Reserve actions, possible political developments, forecasts by economists or whatever else might be the subject du jour. What we see in our holdings, rather, is an assembly of companies that we partly own and that, on a weighted basis, are earning more than 20% on the net tangible equity capital required to run their businesses. We retain our stakes in such companies even when they generate ever-growing excess funds. Our experience is that the long-term financial result from this policy is superior to what we would have achieved by employing those funds in new enterprises.

As you can see, the SBERT model does a good job of identifying the key informational sentences from the original text in each case. The extracted summaries are concise yet cover the main points, and are fluent and coherent since they use the original sentences.

However, being an extractive approach, the summaries may not always capture the high level narrative or flow of ideas in the full text. Some redundancy between sentences is also possible. An abstractive summarization model that can generate novel sentences would be better able to address these limitations and produce human-like summaries, at the cost of some faithfulness to the original text.

Some other potential issues with the SBERT extractive approach:

  • Doesn‘t work well on very short texts like tweets, or ultra long documents like books
  • Extracting individual sentences may lose important context and coreferences
  • The pretrained SBERT models may not generalize well to niche domains like medical or legal text
  • No personalization or interactivity in the summarization process

Despite these caveats, the SBERT extractive summarizer performs very competitively compared to other approaches in benchmarks like CNN/DailyMail and BBC News, while being much more computationally efficient and easier to deploy as a web app compared to huge language models like GPT-3.

Applications and Future Directions

The combination of an accurate, efficient extractive summarization model like SBERT with a user-friendly web interface unlocks a wide range of potential applications, such as:

  • Automatic generation of news article summaries or book reviews
  • Enabling faster analysis of academic papers and technical reports
  • Summarizing meeting notes, earnings calls and business/legal documents
  • Improving the search and discovery experience over large textual databases
  • Voice assistants that can extract key insights from podcasts or Youtube videos
  • Enhanced accessibility for the visually impaired via audio-based summaries

Building on the basic Flask app template covered in this article, you could extend the functionality in many interesting ways:

  • Allowing the user to control the length of the generated summary
  • Adding support for summarizing full webpages or PDF documents in addition to raw text
  • Storing generated summaries in a database for bookmarking/later retrieval
  • Enabling scheduled summarization of articles from RSS feeds or Google Alerts
  • Building a browser extension that auto-summarizes any article or text the user selects
  • Comparing multiple summarization approaches and presenting different summaries to the user

We‘re only scratching the surface of what‘s possible with modern NLP techniques for automatic summarization. As language models continue to advance in their ability to comprehend and generate human-like text, we can expect to see summarization tools that can produce abstractive summaries indistinguishable from those written by humans.

Some exciting developments to keep an eye on:

  • Large language models like GPT-3 and T5 that can be fine-tuned for summarization
  • New neural architectures like transformers and sparse attention optimized for long documents
  • Reinforcement learning approaches to produce more coherent and relevant summaries
  • Personalized and interactive summarization based on user feedback and preferences
  • Multimodal summarization models that incorporate images and video in addition to text
  • Cross-lingual summarization to generate summaries in a different language from source

There are still many open challenges in automatic text summarization, such as evaluating summary quality, ensuring factual consistency, and handling negation and sarcasm. But with the rapid pace of progress in NLP, we can expect to see more intelligent and useful summarization tools emerging to help us tame the ever-growing flood of textual information and effortlessly find the key insights we need.

Conclusion

In this article, we walked through a practical example of building an extractive text summarization tool using a state-of-the-art sentence embedding model, SBERT, and the Flask web framework. With just a few lines of code, we created a working web app that can take in a long article and return a useful summary of the main points.

We explored how SBERT works under the hood to generate semantically meaningful sentence embeddings, and how these can be leveraged to identify the most central and relevant sentences in a document for summarization. We saw examples of the impressive performance of this simple approach on various types of text like news, research papers, and financial reports.

While not a perfect solution, extractive summarization with SBERT provides a fast, scalable, and effective way to get the key information from long text, which can be immensely valuable in this age of information overload. By putting a user-friendly interface on top of powerful NLP models, we can make the ability to get concise, actionable summaries available to anyone at the click of a button.

I encourage you to experiment with the code from this article and try out SBERT-based summarization on your own data. How might a tool like this be useful in your work or everyday life? What other enhancements or features would you add to the basic summarizer app?

The field of NLP and text summarization is rapidly evolving, and it will be exciting to see the new developments and applications that emerge in the coming years. One day soon, we may have AI assistants that can read anything and summarize it better than a human expert! For now, though, I hope this article has given you a taste of the fascinating world of NLP and a practical intro to one of the most promising techniques for making sense of unstructured text.

References and Further Reading

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

You May Like to Read,

Similar Posts