Beginner‘s Guide to Image and Text Similarity (2026)

ByJordan Brown

Introduction

In today‘s digital world, we are surrounded by an ever-growing volume of images and text data. Social media posts, e-commerce product catalogs, medical imaging databases, and satellite imagery are just a few examples of domains where detecting duplicate or highly similar images and text snippets is very useful. Image and text similarity refers to quantifying how "close" or "alike" two images or text strings are to each other.

Some practical applications of image similarity include:

  • Detecting and removing near-duplicate images in datasets
  • Finding visually similar products for recommendations in e-commerce
  • Identifying copyrighted images used without permission
  • Facial recognition to verify a person‘s identity
  • Reverse image search to find related images given a query image

Applications of text similarity include:

  • Catching and filtering spam messages
  • Identifying plagiarism in documents
  • Finding prior art for patents
  • Merging duplicate customer records in databases
  • Retrieving relevant documents for a search query

While humans are exceptionally good at recognizing similar images and texts, it is not always feasible to manually review and compare millions of images or documents. This is where computerized image and text similarity techniques come in. In this guide, we‘ll take a closer look at some popular algorithms for detecting image and text similarity. The focus will be on image similarity, but we‘ll touch upon text similarity too.

Image Similarity Techniques

There are several techniques to programmatically quantify the visual similarity between two images. Each has its own strengths, weaknesses and domain of applicability. Let‘s briefly survey a few prominent ones.

Perceptual Hashing

Perceptual hashing (also called fuzzy hashing) refers to a class of algorithms that generate a fingerprint or hash value for an image, such that visually similar images produce similar hashes. Unlike cryptographic hash functions which aim to magnify even the tiniest difference in input data, perceptual hashes are designed to tolerate minor modifications to an image. Some well-known perceptual hashing algorithms are:

  • pHash (Perceptual Hash): Computes a hash based on the discrete cosine transform (DCT) of an image
  • aHash (Average Hash): Computes a hash based on the mean pixel intensity of an image
  • dHash (Difference Hash): Computes a hash based on pixel intensity gradients
  • wHash (Wavelet Hash): Computes a hash based on wavelet decomposition of an image

The key steps in most perceptual hashing methods are:

  1. Convert the image to grayscale
  2. Resize the image to a small fixed size (e.g. 8×8 pixels)
  3. Compute the hash value based on image properties
  4. Compare the hash values to determine similarity

To compare two perceptual hashes, a distance metric like Hamming distance is typically used. The Hamming distance simply counts the number of bit positions that differ between two hashes. A lower distance indicates higher similarity between the images. Here‘s an illustration of the aHash algorithm:

[Visual illustration of aHash with code snippets]

Perceptual hashing is computationally efficient and can be used to quickly identify near-duplicate images. However, it may not perform well if there are large changes in scale, aspect ratio, rotation or coloring of an image. Perceptual hashes are also sensitive to the choice of image resizing algorithm.

Structural Similarity

The structural similarity index measure (SSIM) is a well-regarded method to quantify the perceived quality degradation between a pristine image and its processed counterpart. It captures the perceived changes in structural information by factoring in luminance, contrast and structure terms. Unlike perceptual hashing which gives a binary outcome (similar or not), SSIM produces a similarity score between -1 and 1. A score of 1 indicates the images are identical.

While SSIM was originally designed for image quality assessment, it can also be used for image similarity. However, it assumes that the images are already aligned and have the same scale, which limits its applicability.

Convolutional Neural Networks

In recent years, convolutional neural networks (CNNs) have revolutionized computer vision and delivered state-of-the-art results in tasks like image classification, object detection and facial recognition. CNNs can also be used for image similarity by comparing the activations in the hidden layers of the network for two input images.

A pretrained CNN like VGG or ResNet is used as a feature extractor by removing the last fully connected layer. Two images are passed through the CNN and their activations at a chosen hidden layer are recorded. The activations are compared using a vector distance metric like cosine distance or Euclidean distance to get a similarity score. Images that are semantically similar produce activations that are close in the learned feature space of the CNN.

Using CNNs for image similarity is powerful since they can learn high-level semantic features of images. However, they require large amounts of training data, are computationally intensive, and can have high memory requirements.

Feature Detection Algorithms

Another approach for image similarity is to detect and match distinctive local features within an image. Some popular feature detection methods are:

  • SIFT (Scale-Invariant Feature Transform)
  • SURF (Speeded Up Robust Features)
  • ORB (Oriented FAST and Rotated BRIEF)

These methods look for keypoints or interest points in an image that are stable under transformations like scaling, rotation and illumination changes. Each keypoint is described by a feature vector or descriptor. To compare two images, their keypoint descriptors are matched based on a distance metric. The number of matching keypoints between two images gives an estimate of their similarity.

Feature detection methods are more robust than perceptual hashing but are computationally more complex. They are a good choice when the images may have changes in viewpoint, scale or orientation.

A Closer Look at Perceptual Hashing

Among the various image similarity techniques, perceptual hashing stands out for its simplicity, computational efficiency and reasonable performance in detecting near-duplicate images. Let‘s dive a bit deeper into how it works.

Most perceptual hashing algorithms follow these steps:

  1. Convert images to grayscale: This discards color information which is often not critical for detecting similarity. It also reduces the size of the data to be hashed.

  2. Resize images to a small fixed size: This step removes high-frequency details and noise from the images. Resizing to a very small size (e.g. 8×8 px) greatly speeds up the hashing while still preserving the essential structural information.

  3. Compute a hash value: The hash value is computed from the pixel intensities of the resized image based on the specific hashing algorithm. The hash is typically a binary string or a sequence of integers.

  4. Compare hashes: To determine if two images are similar, their hash values are compared using a distance metric like Hamming distance. If the distance is below a chosen threshold, the images are considered similar.

Here‘s a code snippet in Python demonstrating the average hash (aHash) algorithm:

import cv2
import numpy as np

def ahash(image, hash_size=8):
    # Resize input image to fixed size 
    resized = cv2.resize(image, (hash_size, hash_size), interpolation=cv2.INTER_AREA)
    # Compute the mean value of image pixels
    mean = resized.mean()
    # Compute the hash bits
    diff = resized > mean
    return sum([2 ** i for (i, v) in enumerate(diff.flatten()) if v])

# Read input images
image1 = cv2.imread(‘image1.jpg‘, cv2.IMREAD_GRAYSCALE) 
image2 = cv2.imread(‘image2.jpg‘, cv2.IMREAD_GRAYSCALE)

# Compute average hashes 
hash1 = ahash(image1)
hash2 = ahash(image2)

# Compare the hashes
if hash1 == hash2:
    print("Images are similar")
else:
    print("Images are not similar")

Let‘s visualize the steps of aHash algorithm. Consider these two input images:

Input Image 1 Input Image 2

After converting to grayscale and resizing to 8×8 pixels:

Grayscale Resized Image 1 Grayscale Resized Image 2

The mean pixel values are:

  • Image 1: 120.8
  • Image 2: 118.3

Subtracting the means and taking the sign, we get the bit matrices:

Image 1 bit matrix:

1 0 0 1 1 0 0 1
0 0 0 1 1 0 0 0 
0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 1 1 0 0 0 1
1 1 1 1 0 1 1 1
1 1 1 1 1 1 1 1

Image 2 bit matrix: 

1 0 0 1 1 0 0 1  
0 0 0 1 1 0 0 0
0 0 0 1 1 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 1
1 1 1 1 0 1 1 1
1 1 1 1 1 1 1 1

Converting the bit matrices to integers, we get the 64-bit hashes:

  • Image 1 hash: 9809771511151908591
  • Image 2 hash: 9809769413839970399

Since the hashes are different, the images are considered not similar by aHash. Indeed, the hashes differ in just 3 bit positions out of 64, which is less than 5%. So aHash is able to detect that the images are in fact very similar despite minor differences.

The choice of hash size is a tradeoff between speed and hash collision probability. A hash size of 8 works well in practice for many applications. The number of possible hashes is 2^64 (~ 10^19) for a 64-bit hash, which makes unintended hash collisions very unlikely.

An advantage of perceptual hashes is that they are generally robust to minor transformations like:

  • Compression artifacts
  • Small amounts of noise
  • Slight cropping or padding
  • Mild blurring
  • Brightness or contrast adjustments

However, their limitations include:

  • Sensitivity to major changes in scale or aspect ratio
  • Inability to detect semantic similarity
  • Susceptibility to adversarial attacks
  • Need for manual tuning of similarity thresholds

Text Similarity Techniques

Detecting similarity or duplication in text has a wide range of applications like spam filtering, plagiarism detection, and record deduplication. Some commonly used text similarity approaches are:

  1. Edit distance: Quantifies similarity based on the number of character edits (insertions, deletions, substitutions) required to transform one string to another. Examples are Levenshtein and Jaro-Winkler distances.

  2. Token-based: The input strings are tokenized (split) into words or n-grams and the overlap between the token sets is measured. Jaccard similarity and cosine similarity of token count vectors are often used.

  3. Sequence matching: Detects common substrings between two strings. The longest common subsequence (LCS) and longest common substring algorithms fall in this category.

  4. Phonetic matching: Strings are converted to a phonetic code and compared based on how they sound rather than how they are spelled. Soundex and Metaphone algorithms are used for matching names.

  5. Word embeddings: Words are mapped to a high-dimensional vector space such that semantically similar words are close together. The vector representations can be learned from a large text corpus using methods like word2vec or GloVe.

The choice of text similarity technique depends on the nature of text data and the desired notion of similarity, i.e., character-level, token-level or semantic similarity.

Practical Considerations

When implementing image or text similarity for a real-world application, there are a few practical aspects to consider:

  • Preprocessing: For images, resizing to a consistent scale and normalizing pixel values can improve similarity results. For text, case normalization, removing punctuation and stop words, and stemming/lemmatization are common preprocessing steps.

  • Similarity thresholds: Choosing the right threshold to decide if two items are similar or not similar is crucial. The threshold depends on the similarity method and the desired precision-recall tradeoff. It can be empirically tuned using a labeled validation set.

  • Combining modalities: Sometimes there is both an image and a text associated with an item, e.g. e-commerce products with photos and descriptions. The similarity scores from both modalities can be combined to improve the overall outcome.

  • Scalability: To find all pairs of similar items in a large dataset, pairwise comparisons can be computationally prohibitive. Locality-sensitive hashing and indexing techniques can be used to scale the similarity search.

  • Evaluation: A sample of similarity predictions should be manually reviewed to estimate the real-world precision and recall. The methodology and similarity thresholds can be iteratively refined based on error analysis.

Conclusion

Image and text similarity are immensely useful in many domains to detect near-duplicate content. In this guide, we covered several techniques for image similarity such as perceptual hashing, structural similarity and neural networks. We also briefly touched upon methods for text similarity like edit distance and token-based similarity.

Perceptual hashing is a good starting point for detecting visually similar images, as it is easy to implement and computationally efficient. For text similarity, the choice of technique depends on whether character-level or semantic similarity is more important for the use case.

Implementing similarity detection requires careful consideration of preprocessing, thresholds, scalability and evaluation methodology. With the right approach, image and text similarity can uncover valuable insights from unstructured data.

To dive deeper into this field, check out these resources:

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

You May Like to Read,

Similar Posts