Image Processing Using Numpy: A Comprehensive Guide with Code Examples
Introduction
As a data scientist or computer vision practitioner, sooner or later you‘ll find yourself needing to process, manipulate, and analyze images using code. Fortunately, the scientific Python ecosystem offers excellent open source tools for working with image data. One library that plays a central role in image processing pipelines is NumPy, the fundamental package for scientific computing in Python.
In this comprehensive tutorial, we‘ll explore the many ways NumPy enables you to perform powerful image processing tasks with concise and efficient code. Starting with the basics of reading and writing image files, we‘ll progress to more advanced operations like filtering, transformations, and feature extraction. Along the way, you‘ll see plenty of code examples to help solidify the concepts. Let‘s dive in!
Loading and Saving Images
Before we can process an image, we first need to load it into memory. NumPy makes this easy with the imread() function, which reads an image from file and returns it as a NumPy array. By default, the image is converted to the uint8 data type with values ranging from 0 to 255.
import numpy as np
from matplotlib import pyplot as plt
img = plt.imread(‘cat.jpg‘)
print(type(img))
print(img.shape)
print(img.dtype)
<class ‘numpy.ndarray‘>
(640, 960, 3)
uint8
As you can see, the loaded image img is a 3D NumPy array with a shape of (640, 960, 3). The first two dimensions correspond to the height and width of the image in pixels, while the third dimension indicates that this is an RGB color image with three channels: red, green, and blue.
To display the loaded image, you can simply use matplotlib‘s imshow() function:
plt.imshow(img)
plt.show()
After performing some image processing operations (which we‘ll cover shortly), you‘ll probably want to save the modified image back to disk. We can achieve this with the imsave() function:
plt.imsave(‘processed_cat.jpg‘, img)
Accessing and Manipulating Pixels
One of the key advantages of representing images as NumPy arrays is that we can efficiently access and manipulate the individual pixels using array indexing and slicing. For example, to get the value of the pixel at row 100, column 200 in our cat image, we can write:
pixel_value = img[100, 200] print(pixel_value)
[156 166 200]In this case, the pixel value is a length-3 array giving the red, green, and blue intensities at that location. To access a single color channel, just add another index:
red_value = img[100, 200, 0] print(red_value)
156
We can also use array slicing to extract rectangular regions of an image. This is known as cropping:
crop = img[50:450, 300:800]
plt.imshow(crop)
plt.show()
The above code extracts a 400×500 pixel crop from the original image, starting at row 50, column 300.
To change the value of a specific pixel, we can use assignment:
img[100, 200] = [255, 0, 0] # Set to bright red
By combining array indexing/slicing with broadcasting operations, we can concisely perform all sorts of useful image manipulations. For instance, to darken an image, we can multiply all pixel values by a constant factor less than 1:
dark_img = img * 0.5
plt.imshow(dark_img)
plt.show()
Flipping and Rotating Images
Flipping an image horizontally or vertically is as simple as reversing the order of the rows or columns using a slice with a negative step:
flipped_h = img[:, ::-1] flipped_v = img[::-1] flipped_hv = img[::-1, ::-1]
plt.subplot(2, 2, 1)
plt.imshow(img)
plt.title(‘Original‘)
plt.subplot(2, 2, 2)
plt.imshow(flipped_h)
plt.title(‘Flipped Horizontally‘)
plt.subplot(2, 2, 3)
plt.imshow(flipped_v)
plt.title(‘Flipped Vertically‘)
plt.subplot(2, 2, 4)
plt.imshow(flipped_hv)
plt.title(‘Flipped Horizontally & Vertically‘)
plt.tight_layout()
plt.show()
To rotate an image by 90 degrees, we can use the rot90() function from NumPy. We can also specify the number of times to rotate by 90 degrees:
rotated_90 = np.rot90(img)
rotated_180 = np.rot90(img, 2)
rotated_270 = np.rot90(img, 3)
For arbitrary rotation angles, the scipy.ndimage.rotate() function can be used.
Image Filtering
Filtering is a fundamental image processing operation that involves applying a kernel or convolution matrix to each pixel and its neighbors. This can achieve effects like blurring, sharpening, edge detection, and more.
Blurring an image is useful for removing noise or small details. It can be done by convolving the image with a 2D Gaussian kernel. Here‘s an example using the scipy.ndimage.gaussian_filter() function:
from scipy.ndimage import gaussian_filter
blurred = gaussian_filter(img, sigma=5)
plt.subplot(1, 2, 1)
plt.imshow(img)
plt.title(‘Original‘)
plt.subplot(1, 2, 2)
plt.imshow(blurred)
plt.title(‘Blurred‘)
plt.tight_layout()
plt.show()
The sigma parameter controls the amount of blurring – higher values will result in a more blurred image.
On the flip side, we can sharpen an image by subtracting a blurred version of the image from the original:
sharpened = img – gaussian_filter(img, sigma=3)
plt.subplot(1, 2, 1)
plt.imshow(img)
plt.title(‘Original‘)
plt.subplot(1, 2, 2)
plt.imshow(sharpened)
plt.title(‘Sharpened‘)
plt.tight_layout()
plt.show()
Edge detection is another classical image processing task that highlights boundaries and discontinuities in an image. The Sobel filter is often used for this purpose:
from scipy.ndimage import sobel
edges_x = sobel(img, axis=0)
edges_y = sobel(img, axis=1)
edges = np.hypot(edges_x, edges_y)
plt.subplot(1, 3, 1)
plt.imshow(edges_x, cmap=‘gray‘)
plt.title(‘Edges (X direction)‘)
plt.subplot(1, 3, 2)
plt.imshow(edges_y, cmap=‘gray‘)
plt.title(‘Edges (Y direction)‘)
plt.subplot(1, 3, 3)
plt.imshow(edges, cmap=‘gray‘)
plt.title(‘Edges (Magnitude)‘)
plt.tight_layout()
plt.show()
Here we compute the Sobel gradients along the x and y directions separately, then combine them to get the overall edge magnitude.
Thresholding and Binarization
Thresholding is the simplest method of segmenting an image into regions of interest. The idea is to classify each pixel as either belonging to the foreground or background based on its intensity value.
In the simplest case of global thresholding, we choose a single cutoff value. All pixels with intensities below the threshold are set to black (0), while pixels above the threshold are set to white (255). This produces a binary image:
threshold = 127
binary = np.where(img > threshold, 255, 0).astype(np.uint8)
plt.subplot(1, 2, 1)
plt.imshow(img, cmap=‘gray‘)
plt.title(‘Original‘)
plt.subplot(1, 2, 2)
plt.imshow(binary, cmap=‘gray‘)
plt.title(‘Binarized‘)
plt.tight_layout()
plt.show()
The where() function is used here to apply the threshold to each pixel. The astype(np.uint8) call at the end ensures the resulting binary image has the correct data type.
More sophisticated thresholding techniques like Otsu‘s method automatically determine an optimal threshold value:
from skimage.filters import threshold_otsu
otsu_thresh = threshold_otsu(img)
binary = np.where(img > otsu_thresh, 255, 0).astype(np.uint8)
Histogram Analysis and Contrast Stretching
A histogram shows the distribution of pixel intensities in an image. It can help identify issues like over/underexposure and low contrast. We can easily compute and plot a histogram using NumPy and Matplotlib:
histogram = np.histogram(img, bins=256, range=(0, 256))
plt.figure(figsize=(10, 4))
plt.subplot(1, 2, 1)
plt.imshow(img, cmap=‘gray‘)
plt.title(‘Image‘)
plt.axis(‘off‘)
plt.subplot(1, 2, 2)
plt.plot(histogram[1][:-1], histogram[0], lw=2)
plt.title(‘Histogram of Pixel Intensities‘)
plt.xlabel(‘Pixel Intensity‘)
plt.ylabel(‘Count‘)
plt.tight_layout()
plt.show()
If an image has low contrast, its histogram will be bunched up in a narrow range. We can improve the contrast by stretching the histogram to span the full range of intensities. A simple way to do this is:
def contrast_stretch(img):
in_min = np.min(img)
in_max = np.max(img)
out_min = 0.0
out_max = 255.0
out = img - in_min
out *= ((out_min - out_max) / (in_min - in_max))
out += in_min
return outstretched = contrast_stretch(img)
plt.subplot(2, 2, 1)
plt.imshow(img, cmap=‘gray‘)
plt.title(‘Original‘)
plt.axis(‘off‘)
plt.subplot(2, 2, 2)
_ = plt.hist(img.flatten(), 256, [0, 256])
plt.title(‘Original Histogram‘)
plt.subplot(2, 2, 3)
plt.imshow(stretched, cmap=‘gray‘)
plt.title(‘Contrast Stretched‘)
plt.axis(‘off‘)
plt.subplot(2, 2, 4)
_ = plt.hist(stretched.flatten(), 256, [0, 256])
plt.title(‘New Histogram‘)
plt.tight_layout()
plt.show()
Morphological Operations
Morphological operations process images based on their shapes. They‘re commonly used for tasks like removing small objects or holes, separating touching objects, and finding intensity bumps.
The two fundamental morphological operations are erosion and dilation. Erosion shrinks bright regions and enlarges dark regions, while dilation expands bright regions and shrinks dark regions.
Here‘s an example of using erosion to remove salt noise from an image:
from scipy.ndimage import binary_erosion
noisy = img_binary | (np.random.random(img_binary.shape) < 0.1)
eroded = binary_erosion(noisy)
To remove pepper noise, we can use dilation:
from scipy.ndimage import binary_dilation
noisy = img_binary & (np.random.random(img_binary.shape) < 0.9)
dilated = binary_dilation(noisy)
Two other important morphological operations are opening and closing. Opening is erosion followed by dilation, while closing is dilation followed by erosion. Opening can remove small bright spots, while closing can remove small dark holes.
from scipy.ndimage import binary_opening, binary_closing
opened = binary_opening(img_binary)
closed = binary_closing(img_binary)
Working with Color Spaces
So far we‘ve been working with grayscale and RGB images. However, there are many other color spaces that are useful for different tasks. Two common ones are HSV (hue, saturation, value) and Lab (perceptually uniform).
We can convert between color spaces using functions from skimage:
from skimage.color import rgb2hsv, rgb2lab, hsv2rgb, lab2rgb
hsv = rgb2hsv(img)
lab = rgb2lab(img)
recovered_rgb_from_hsv = hsv2rgb(hsv)
recovered_rgb_from_lab = lab2rgb(lab)
The HSV color space is useful for tasks like color-based segmentation, while Lab is often used for color transfer between images.
Getting Creative with Image Effects
By combining the techniques we‘ve learned, we can create all sorts of interesting artistic effects. For example, here‘s how to create a "pencil sketch" effect:
from scipy.ndimage import median_filter
inverted = 255 – img
blurred = median_filter(inverted, size=5)
sketch = blurred * 1.2 – 70
plt.subplot(1, 2, 1)
plt.imshow(img)
plt.title(‘Original‘)
plt.axis(‘off‘)
plt.subplot(1, 2, 2)
plt.imshow(sketch, cmap=‘gray‘)
plt.title(‘Pencil Sketch‘)
plt.axis(‘off‘)
plt.tight_layout()
plt.show()
Or a "painted" effect using bilateral filtering and color quantization:
from skimage.filters import median
from skimage.color import rgb2lab, lab2rgb
from skimage.transform import rescale
small = rescale(img, 0.25)
lab = rgb2lab(small)
lab[:, :, 0] = median(lab[:, :, 0], np.ones((5, 5)))
lab[:, :, 1] = median(lab[:, :, 1], np.ones((5, 5)))
lab[:, :, 2] = median(lab[:, :, 2], np.ones((5, 5)))
painted = lab2rgb(lab)
plt.subplot(1, 2, 1)
plt.imshow(img)
plt.title(‘Original‘)
plt.axis(‘off‘)
plt.subplot(1, 2, 2)
plt.imshow(painted)
plt.title(‘Painted‘)
plt.axis(‘off‘)
plt.tight_layout()
plt.show()
The possibilities are endless – I encourage you to experiment and come up with your own creative effects!
Conclusion
In this article, we‘ve only scratched the surface of what‘s possible with image processing in Python using NumPy. Despite its simplicity, NumPy provides an incredibly powerful and flexible foundation for loading, manipulating, and transforming image data.
Of course, NumPy isn‘t the only tool in the image processing toolbox. Other indispensable libraries include:
- Pillow: Convenient utilities for reading/writing image files in many formats
- OpenCV: Industrial-strength computer vision library with advanced capabilities
- scikit-image: Ever-expanding collection of image processing algorithms
- Matplotlib: The go-to plotting library for exploratory analysis and visualization
With NumPy at the core and these libraries in support, you‘re well-equipped to tackle almost any image processing task, whether for hobby projects, scientific research, or production systems. So choose an image, start exploring, and see what insights you can uncover. Happy processing!
