Create Your Own Snapchat Filter with OpenCV – Give Hermione Granger Magic Sunglasses!

ByJordan Brown

Who doesn‘t love playing with wacky filters that can instantly transform your face? Whether it‘s puppy dog ears, a pirate‘s eye patch, a crown of flowers, or that ubiquitous beauty filter, apps like Snapchat and Instagram have made augmented selfies a huge trend. But did you know you can create your very own custom filter completely from scratch using OpenCV?

In this post, we‘ll walk through how to use facial feature detection in OpenCV to give everyone‘s favorite brilliant witch, Hermione Granger, a stylish pair of sunglasses. Accio cool code!

What is OpenCV?

Before we jump into building our filter, let‘s first understand what OpenCV is. OpenCV (Open Source Computer Vision Library) is an open source library of programming functions aimed at real-time computer vision. Developed by Intel and first released in 2000, it has become an essential tool for image processing and computer vision tasks.

Some key capabilities of OpenCV include:

  • Image and video input/output, display
  • Object, face and feature detection
  • Image filtering, manipulation and transformations
  • Image stitching to create panoramas
  • 3D reconstruction
  • Machine learning, including clustering and classification algorithms

OpenCV has interfaces for multiple languages including C++, Python, and Java. Here we‘ll be using OpenCV in Python, which provides a concise and readable way to access its functionality. The library is widely used in both academia and industry for applications like photo editing, autonomous vehicles, facial recognition, and augmented reality.

Haar Cascade Classifiers for Face Detection

To place Hermione‘s sunglasses, we first need to detect where her eyes are located in the image. For this, we‘ll use a Haar Cascade Classifier, a machine learning based approach for object detection.

Haar Cascade is a algorithm that can detect objects in images, regardless of their location and scale. It was originally developed by Paul Viola and Michael Jones in 2001. The algorithm uses a set of positive images (with the desired object) and negative images (without the object) to train a cascade function. The trained classifier is then applied to regions of an input image and returns 1 if the region is likely to contain the object and 0 otherwise.

Key concepts of the Haar Cascade algorithm include:

  • Haar-like features: Simple rectangular features that calculate the difference in intensity between adjacent regions. For example, detecting an eye might use a feature that compares the intensity of the eye region (dark) to that of the upper cheek (light).

  • Integral image: An optimization technique that allows features to be computed very quickly. Each pixel in the integral image contains the sum of all pixels above and to the left of it in the original image.

  • AdaBoost: A machine learning algorithm that combines many "weak" classifiers (that are only slightly better than random guessing) into a "strong" classifier.

  • Cascade: The classifiers are organized into stages, where each stage has a certain number of features. If a region fails at any stage, it is immediately discarded as "not face". Regions that pass all stages are classified as "face". This structure allows non-face regions to be quickly discarded, improving detection speed.

OpenCV provides pre-trained Haar cascades as XML files that can be used for common object detection tasks like faces, eyes, smiles, etc. Here we‘ll use the "haarcascade_eye.xml" file to detect Hermione‘s eyes. You can find this file, along with other useful Haar cascades, in OpenCV‘s GitHub repository.

Now that we understand the core concepts, let‘s dive into the code and give Hermione those magical shades!

Import Libraries and Read Image

We‘ll start by importing the necessary Python libraries: NumPy for numerical computing, Matplotlib for plotting, and of course cv2 for OpenCV.

import numpy as np 
import matplotlib.pyplot as plt
import cv2

Next, let‘s read in the image of Hermione using Matplotlib‘s imread() function. We‘ll also create a copy of the image to use later when overlaying the sunglasses.

hermione = plt.imread(‘hermione.jpg‘)
hermione_glasses = hermione.copy()


Image credit: Harry Potter and the Order of the Phoenix, Warner Bros. Pictures, 2007

We can check the dimensions of the image using NumPy‘s shape attribute:

print(hermione.shape)
(500, 408, 3)

This tells us that the loaded image is 500 pixels tall, 408 pixels wide, with 3 color channels (RGB).

Detect Hermione‘s Eyes

Now we‘re ready to detect Hermione‘s eyes using the Haar cascade classifier. First, we load the pre-trained cascade from the XML file:

eye_cascade = cv2.CascadeClassifier(‘haarcascade_eye.xml‘)

Then we use the detectMultiScale() method to find the eye regions in the image:

eyes = eye_cascade.detectMultiScale(hermione)
print(eyes)

[[151 183  85  85]
 [259 183  85  85]]

The output is an array of detections, where each row represents a detected eye. The numbers are the x, y coordinates of the top-left corner, followed by the width and height of the detection rectangle.

Since we want to place the sunglasses over both eyes, we‘ll calculate the overall eye region by taking the minimum x, minimum y, maximum x, and maximum y of the two detections.

x, y, w, h = eyes[0] 
for (x2, y2, w2, h2) in eyes[1:]:
    x = min(x, x2)
    y = min(y, y2)
    w = max(w, w2, x+w2-x)  
    h = max(h, h2, y+h2-y)

Let‘s plot a rectangle around the detected eye region to verify it:

cv2.rectangle(hermione, (x,y), (x+w,y+h), (255,255,255), 2)
plt.imshow(hermione)

The white rectangle confirms that we‘ve identified the correct eye region. Alohomora, we‘re in!

Resize Sunglasses and Overlay

With the eye region locked in, we can now read in our sunglasses image and resize it to fit Hermione‘s face.

sunglasses = plt.imread(‘sunglasses.png‘)
print(sunglasses.shape)
(200, 446, 4)

Notice that the sunglasses image has a 4th dimension (the alpha channel) which controls the transparency. We‘ll use that later to blend the glasses realistically.

To resize the sunglasses, we calculate a rescale factor based on the width of the eye detection compared to the width of the glasses:

rescale_factor = w / sunglasses.shape[1] 
sunglasses_resized = cv2.resize(sunglasses, (w, int(sunglasses.shape[0] * rescale_factor)))

Finally, we can overlay the resized sunglasses onto Hermione‘s face using array slicing. The .shape() method returns the dimensions of the sunglasses image, which we use to determine the overlay region on the face. The alpha channel of the sunglasses image is used as a mask to blend the pixels.

overlay_img = hermione_glasses[y:y+sunglasses_resized.shape[0], x:x+sunglasses_resized.shape[1]]
overlay_glasses = sunglasses_resized[:,:,:3]  
overlay_alpha = sunglasses_resized[:,:,3:] / 255.0
overlay_img[:] = (1.0 - overlay_alpha) * overlay_img + overlay_alpha * overlay_glasses

Let‘s break that down:

  • overlay_img is the slice of the hermione_glasses image where the sunglasses will be placed
  • overlay_glasses is the RGB channels of the resized sunglasses image
  • overlay_alpha is the normalized (0 to 1) alpha channel of the resized sunglasses
  • For each pixel, we do a weighted sum of the original pixel value and glasses pixel value based on the alpha

Time for the grand reveal – plot the final image with Hermione‘s slick new shades!

plt.imshow(hermione_glasses)

Wingardium leviosa! There you have it – Hermione Granger rocking some magical sunglasses worthy of the brightest witch of her age, all thanks to OpenCV.

Learn More

I hope this post has given you a taste of the exciting world of computer vision and the fun you can have with OpenCV and Python. The techniques we used here for face detection and augmentation are just the tip of the iceberg of what‘s possible. You can use a similar approach to add all kinds of creative overlays – from animal ears to pirate hats, the only limit is your imagination!

Challenge yourself to try this out on your own images. Play around with different Haar cascades (OpenCV has them for things like smiles, hands, full bodies) and overlay PNGs. Share your wackiest creations with the world!

If you want to dive deeper into the magic of OpenCV, here are some great resources to check out:

Happy coding, and may your filters be ever in your favor!

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

You May Like to Read,

Similar Posts