# Face Detection in Images using the MediaPipe Library - Canonical: https://33rdsquare.com/detect-the-faces-in-the-image-using-the-mediapipe-library/ - Published: 2024-09-03 - Author: Jordan Brown - Categories: [Artificial Intelligence & Machine Learning & ChatGPT](https://33rdsquare.com/category/tech/ai/) --- Face detection is a key first step for many exciting applications in computer vision, from face recognition for biometrics to emotion detection for human-computer interaction. While there are many open source and commercial options for face detection, Google‘s MediaPipe library stands out for its ease of use, strong performance, and ability to run in real-time on a variety of platforms from servers to edge devices. In this tutorial, we‘ll walk through how to use MediaPipe to quickly and accurately detect faces in images using Python. We‘ll cover the face detection pipeline, look under the hood at the models and algorithms used, and show step-by-step code examples. By the end, you‘ll be equipped to apply MediaPipe face detection to your own projects. Let‘s dive in! ## What is MediaPipe? MediaPipe is an open-source machine learning library from Google that provides a collection of pipelines and models for building multimodal (vision, audio, etc.) AI systems. The library is cross-platform and can run on servers, desktops, mobile and edge devices. Some key features of MediaPipe include: - Facial landmark detection for localizing key points on the face - Hand tracking to detect and track the 21 3D landmarks of a hand from just a single frame - Pose estimation for detecting 33 3D landmarks on the body - Hair segmentation for selfie effects and augmented reality - Object detection for localizing and classifying multiple objects in an image In this tutorial, we‘ll focus specifically on MediaPipe‘s Face Detection capabilities. The face detector is able to localize faces in an image, determine the bounding box and key landmarks, and track faces across frames. It provides the foundation for building face-based AI applications. ## The MediaPipe Face Detection Pipeline At the core of MediaPipe‘s face detection pipeline is the BlazeFace face detection model. BlazeFace is a lightweight and fast model designed for real-time face detection on mobile devices. It uses a modified MobileNetV1/V2 backbone and a multitask loss to achieve an impressive balance of speed and accuracy. The overall MediaPipe face detection pipeline consists of two models working in tandem: 1. A face detection model for coarse localization of faces in the image. This uses the BlazeFace backbone to generate a set of face bounding box proposals. 2. A face landmark model for detecting 6 key points on each detected face – the corners of the eyes, the tip of the nose, the mouth center, and the ear tragions. These landmarks help to more precisely localize the face and enable applications like facial filters. When an image is passed to the pipeline, it is first resized and padded to a standard size. The face detection model then runs inference to generate face proposals. Non-maximum suppression is applied to eliminate overlapping detections. Each face proposal is then passed to the landmark model to predict the 6 face key points. Finally, the face detections can be visualized by drawing the bounding boxes and landmarks on the image. The MediaPipe face detection pipeline is highly optimized and can run in real-time. On a desktop CPU, speeds of 100-200 FPS are possible, while mobile devices can still achieve 30-50 FPS. This enables a wide variety of interactive applications. ## Using MediaPipe Face Detection Now that we have an understanding of the MediaPipe face detection pipeline, let‘s walk through an example of using it to find faces in an image with Python. The code is quite straightforward. First, install the MediaPipe library: ``` !pip install mediapipe ``` Next, let‘s import the required packages: ``` import cv2 import mediapipe as mp import matplotlib.pyplot as plt ``` Now we‘ll load an example image and visualize it: ``` sample_img = cv2.imread(‘sample.jpg‘) plt.figure(figsize=[10,10]) plt.title("Sample Image") plt.imshow(sample_img[:,:,::-1]) ``` To use MediaPipe for face detection, we first create an instance of the `FaceDetection` class, passing in the model type and minimum detection confidence threshold: ``` face_detection = mp.solutions.face_detection.FaceDetection( model_selection=1, min_detection_confidence=0.5) ``` Here we‘re using model type 1 which is a heavier but more accurate version of BlazeFace suited for cases where faces may be farther away (up to 5 meters). The default model 0 is faster and works well for faces within 2 meters of the camera. The `min_detection_confidence` parameter controls the minimum confidence threshold for a face detection to be considered. Higher values will reduce false positives but may miss some faces, while lower values will detect more faces but with more false positives. The default of 0.5 is a good tradeoff. To run face detection on the image, we pass it to the `process` method: ``` results = face_detection.process(sample_img) ``` This runs the face detection pipeline and returns an object with a `detections` field containing a list of detected faces. Each detection includes the bounding box, detection confidence, and face landmarks. We can visualize the results by drawing the face detections on the image: ``` if results.detections: annotated_image = sample_img.copy() for detection in results.detections: mp_drawing.draw_detection(annotated_image, detection) plt.figure(figsize=[10,10]) plt.imshow(annotated_image[:,:,::-1]) ``` Here‘s the result: ![Face detection results](https://33rdsquare.com/face_detection_result.jpg) As we can see, MediaPipe has successfully found the face in the image, drawn a bounding box around it, and located the key facial landmarks like eyes, nose, mouth, and ears. We can access the numerical results as well, for example to get the face bounding box and detection confidence: ``` for detection in results.detections: print(f‘Confidence: {round(detection.score[0]*100, 2)}%‘) bbox = detection.location_data.relative_bounding_box print(f‘Bounding box:‘) print(f‘ Top: {bbox.ymin}‘) print(f‘ Bottom: {bbox.ymax}‘) print(f‘ Left: {bbox.xmin}‘) print(f‘ Right: {bbox.xmax}‘) ``` This prints out: ``` Confidence: 97.89% Bounding box: Top: 0.33422425 Bottom: 0.7027026 Left: 0.36862865 Right: 0.6099164 ``` The bounding box coordinates are returned as relative values between 0 and 1, which can be scaled to pixel values based on the dimensions of the image. We can also access the face landmark coordinates: ``` print(f‘Face landmarks:‘) for landmark in detection.location_data.relative_keypoints: print(f‘ {mp.solutions.face_detection.FaceKeyPoint(landmark).name}:‘) print(f‘ x: {landmark.x}‘) print(f‘ y: {landmark.y}‘) ``` This outputs: ``` Face landmarks: RIGHT_EYE: x: 0.46660355 y: 0.46241686 LEFT_EYE: x: 0.54242706 y: 0.4556241 NOSE_TIP: x: 0.49867052 y: 0.5432929 MOUTH_CENTER: x: 0.50870573 y: 0.6034963 RIGHT_EAR_TRAGION: x: 0.38539445 y: 0.5123174 LEFT_EAR_TRAGION: x: 0.5803514 y: 0.5081731 ``` The face landmark coordinates are also returned as relative values, which can be scaled to pixel coordinates if needed. With these face bounding boxes and landmarks, we can proceed to all sorts of interesting applications, like recognizing who the face belongs to, classifying their emotional state, applying real-time facial filters, and more. ## Performance So how well does MediaPipe face detection actually perform? Let‘s take a look at some benchmarks. On the popular FDDB face detection dataset ([http://vis-www.cs.umass.edu/fddb/](http://vis-www.cs.umass.edu/fddb/)), BlazeFace achieves an impressive 95.3% true positive rate at 1000 false positives. This is comparable to much heavier and slower models like RetinaFace. In terms of speed, MediaPipe face detection can run at up to 1000 FPS on high-end desktop GPUs. More practically, speeds of 100-200 FPS are typical on a desktop CPU, while mobile devices can achieve real-time speeds of 30-50 FPS. This enables interactive applications even on resource-constrained edge devices. Compared to other open-source face detection models, MediaPipe provides an outstanding balance of accuracy and speed that is hard to beat. It outperforms classic models like Haar Cascades and HOG-based detectors. And while it may not match the absolute peak accuracy of massive models like RetinaFace, it comes very close while being orders of magnitude faster. ## Conclusion In this tutorial, we‘ve seen how to use Google‘s MediaPipe library to easily perform fast and accurate face detection in images using Python. With just a few lines of code, we can localize faces, determine facial landmarks, and enable all sorts of exciting face-based computer vision applications. The combination of a state-of-the-art BlazeFace model and a carefully optimized inference pipeline makes MediaPipe face detection a compelling choice for anything from interactive mobile apps to large-scale batch inference in the cloud. It achieves an impressive balance of accuracy and speed. Equipped with this face detection foundation, you‘re now ready to build your own face-based AI systems! Some ideas to try: - Face recognition to identify individuals - Facial expression classification for emotion detection - Facial landmark tracking for AR filters and effects - Attention tracking based on eye gaze - Face clustering to group similar faces The possibilities are endless! I encourage you to explore the MediaPipe GitHub page ([https://github.com/google/mediapipe](https://github.com/google/mediapipe)) for more resources and ideas. And if you build something interesting, please share it with the world! I hope you‘ve enjoyed this tutorial and now feel empowered to use MediaPipe face detection in your own projects. It‘s an awesome tool to have in your computer vision toolbox. Happy detecting! --- Source: [Face Detection in Images using the MediaPipe Library](https://33rdsquare.com/detect-the-faces-in-the-image-using-the-mediapipe-library/)