# A Comprehensive Guide to Optical Character Recognition \(OCR\) with Tesseract, OpenCV and Python

- Canonical: https://33rdsquare.com/optical-character-recognitionocr-with-tesseract-opencv-and-python/
- Published: 2024-09-03
- Author: Jordan Brown
- Categories: [Artificial Intelligence & Machine Learning & ChatGPT](https://33rdsquare.com/category/tech/ai/)

---

## What is Optical Character Recognition (OCR)?

Optical Character Recognition, or OCR for short, is a technology that enables conversion of images of typed, handwritten or printed text into machine-encoded text. In other words, OCR allows computers to "read" text in images, making that text digitally accessible as machine-readable data that can be searched, edited, stored and analyzed.

At its core, OCR works by analyzing the patterns of dark and light in an image to identify shapes that correspond to letters and numbers, and then converting those into digital text. Modern OCR systems leverage machine learning and deep learning algorithms to achieve high levels of speed and accuracy in this image-to-text conversion process.

## Applications and Business Use Cases for OCR

The ability to extract text from images opens up a wide range of powerful applications for OCR across industries. Some key examples include:

- Digitizing printed documents and records to enable full-text search, reduce physical storage costs, and integrate with digital workflow systems. Common applications include digitizing legal contracts, medical records, academic articles, financial statements and more.
- Automating data entry from forms and other structured documents to save time and improve accuracy. For example, OCR can be used to automatically populate databases with data from scanned paper forms, PDFs, invoices, receipts, etc.
- Enabling text-to-speech for images, which is very useful for accessibility. OCR can be used to extract the text from images and then that text can be read aloud using text-to-speech engines.
- Indexing text in images for search engines. While search technology can easily index digital text, it cannot directly access text locked in image files. OCR enables that text to be extracted and then indexed to make images searchable.
- Analyzing text from images for business intelligence. OCR provides a way to quantify and analyze unstructured information that would otherwise be inaccessible, such as customer feedback from handwritten comment cards.
- Building augmented reality and robotics applications by enabling systems to read and interact with text in the physical world, such as signs and labels.

As you can see, OCR has applications that span domains from document management to analytics to accessibility. In our increasingly visual digital world, OCR is a key enabler for building systems that can access and make use of the huge amount of valuable information encoded in image format.

## Getting Started with OCR in Python using Tesseract and OpenCV

Python is an ideal language for working with OCR thanks to its extensive ecosystem of powerful open source libraries. In this tutorial, we‘ll be using two key libraries:

1. **Tesseract** – Tesseract is an open source OCR engine that was originally developed by Hewlett-Packard in the 1980s and was then released as open source in 2005. It is currently maintained by Google and is widely considered the best open source OCR engine available. Tesseract supports over 100 languages, can detect both vertical and horizontal text, and is highly configurable.
2. **OpenCV** – OpenCV (Open Source Computer Vision Library) is an open source library of programming functions for real-time computer vision. It provides a wide range of features for image and video processing that are extremely useful for pre-processing images to enhance OCR accuracy.

Let‘s walk through the process of setting up these tools and using them to perform OCR on an image using Python.

### Step 1: Install Tesseract

First, you‘ll need to install the Tesseract OCR engine on your system if you don‘t already have it. The installation process varies depending on your operating system:

- On Mac, you can install Tesseract using Homebrew by running: `brew install tesseract`
- On Linux, you can install Tesseract using apt-get: `sudo apt install tesseract-ocr`
- On Windows, you‘ll need to download the installer from the Tesseract project page on GitHub.

### Step 2: Install Python and Required Libraries

Next, make sure you have Python installed. This tutorial will use Python 3.

Then, install the Pillow, OpenCV, and pytesseract libraries using pip:

`pip install pillow
pip install opencv-python
pip install pytesseract`

### Step 3: Basic OCR with Pytesseract

With the setup out of the way, let‘s see how simple it is to use the pytesseract library to extract text from an image file in Python. Here‘s the code:

`from PIL import Image
import pytesseract`

img = Image.open(‘sample.jpg‘)
 text = pytesseract.image_to_string(img)
 print(text)

This code simply opens an image file called "sample.jpg" using the Pillow library, and then passes that image to pytesseract‘s image_to_string function which performs OCR on the image and returns the extracted text. Finally, we print out the text.

That‘s it! With just a few lines of Python code we‘re able to extract text from an image using Tesseract OCR. However, often the quality of the input image isn‘t ideal, which can lead to errors or inaccuracies in the OCR output. That‘s where OpenCV comes in.

### Step 4: Improving OCR Accuracy with OpenCV

OpenCV provides a range of image processing techniques that we can use to pre-process our input images to enhance the accuracy of Tesseract‘s OCR. Here are a few key techniques:

1. **Convert to Grayscale** – Converting a color image to grayscale can help simplify the image and make the text stand out more clearly.

`import cv2
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)`

1. **Apply Thresholding** – Thresholding can help separate the text from the background of the image. OpenCV provides several thresholding techniques, but a simple binary threshold is often effective.

`thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]`

1. **Noise Removal** – Removing noise from the image can help clean up small specks or dots that might be interpreted as characters by the OCR engine. One simple technique is to apply a median blur to the image.

`blur = cv2.medianBlur(thresh, 3)`

1. **Dilation and Erosion** – Dilation can help connect broken parts of characters, while erosion can help separate characters that are touching. These morphological transformations can help improve the clarity of the characters for the OCR engine.

`kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
dilate = cv2.dilate(blur, kernel, iterations=1)
erode = cv2.erode(dilate, kernel, iterations=1)`

Putting this all together, here‘s what the code might look like:

`import cv2
import pytesseract`

img = cv2.imread(‘sample.jpg‘)

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
 thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
 blur = cv2.medianBlur(thresh, 3)

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
 dilate = cv2.dilate(blur, kernel, iterations=1)
 erode = cv2.erode(dilate, kernel, iterations=1)

text = pytesseract.image_to_string(erode)
 print(text)

Experiment with these and other techniques to see what works best for your specific use case and the types of images you‘re working with.

## Advanced OCR Applications

With the basics of OCR in Python under your belt, let‘s explore a couple more advanced applications.

### Building Word Clouds from Image Text

Word clouds are a popular way to visually represent the frequency of words in a text. We can use OCR to extract the text from an image, and then use that text to generate a word cloud. Here‘s how:

`import cv2
import pytesseract
from wordcloud import WordCloud
import matplotlib.pyplot as plt`

img = cv2.imread(‘sample.jpg‘)
 text = pytesseract.image_to_string(img)

wordcloud = WordCloud().generate(text)

plt.imshow(wordcloud, interpolation=‘bilinear‘)
 plt.axis("off")
 plt.show()

This code uses the wordcloud library to generate a word cloud from the text extracted by OCR, and then uses matplotlib to display the word cloud image.

### Creating an OCR-Powered Text-to-Speech Engine

We can also combine OCR with text-to-speech to create a system that can read aloud the text in an image. Here‘s a simple example using the pyttsx3 library for text-to-speech:

`import cv2
import pytesseract
import pyttsx3`

img = cv2.imread(‘sample.jpg‘)
 text = pytesseract.image_to_string(img)

engine = pyttsx3.init()
 engine.say(text)
 engine.runAndWait()

This code initializes a pyttsx3 engine, passes the OCR text to the engine‘s say method, and then runs the engine to speak the text aloud.

## Alternatives to Tesseract

While Tesseract is the most popular open source OCR engine, there are other alternatives worth considering depending on your specific needs:

- **Google Cloud Vision API** – A commercial OCR service that leverages Google‘s AI and machine learning capabilities. It offers very high accuracy but requires an internet connection and has usage costs.
- **Amazon Textract** – Another commercial OCR service, provided as part of AWS. Textract is specifically designed to extract data from forms and tables.
- **EasyOCR** – A Python library for OCR that aims to be a more flexible alternative to Tesseract. It supports over 70 languages and has a simple interface.
- **Keras-OCR** – A package that provides a high-level API for building OCR systems using the Keras deep learning library. It allows training custom OCR models on your own datasets.

## The Future of OCR

As AI and machine learning continue to advance, we can expect OCR technology to become even more accurate, efficient and flexible in the coming years. Some key areas of development include:

- **Improved accuracy on handwritten and cursive text** – Traditional OCR engines have struggled with the variability of handwritten text, but new deep learning approaches are making significant strides in this area.
- **Better support for low-quality and noisy images** – Techniques like super-resolution and de-noising using deep neural networks can help improve OCR performance on low-quality images.
- **More efficient mobile and edge implementations** – As more computing moves to mobile devices and edge computing nodes, we‘ll see more efficient OCR implementations that can run in real-time and with limited resources.
- **Tighter integration with other AI technologies** – OCR will increasingly be combined with other AI technologies like natural language processing, machine translation, and computer vision to enable more sophisticated applications.

## Wrapping Up

OCR is a powerful technology with a wide range of applications across industries. With the rise of deep learning and the increasing availability of powerful open source tools like Tesseract and OpenCV, it‘s easier than ever to get started with OCR and build applications that can extract valuable information from images.

In this article, we‘ve covered the basics of OCR, walked through a hands-on tutorial using Tesseract and OpenCV with Python, explored some advanced applications, and discussed alternatives and future directions for OCR technology.

OCR is a complex field and there‘s always more to learn. But armed with the foundations covered in this guide, you‘re well on your way to unlocking the potential of OCR in your own projects. Happy coding!

---

Source: [A Comprehensive Guide to Optical Character Recognition \(OCR\) with Tesseract, OpenCV and Python](https://33rdsquare.com/optical-character-recognitionocr-with-tesseract-opencv-and-python/)