Hacking Human Vision: Creating Mind-Bending Optical Illusions with Python

Optical illusions have captivated humans for centuries, from ancient Greek trompe l‘oeil frescos to brain-teasing street art and viral internet memes. These clever images exploit glitches in our visual processing system to create perceptions that defy reality. Straight lines appear curved, static patterns seem to move, and identical colors look distinct.

As an AI and machine learning practitioner, I find illusions fascinating for more than just their trippy entertainment value. They illuminate core mechanisms of biological and artificial intelligence in action. The same shortcuts our brains take to efficiently process visual information – and occasionally get fooled – are echoed in the quirks of convolutional neural networks and computer vision algorithms.

In this guide, we‘ll explore the science of optical illusions and the art of creating them with Python. Along the way, we‘ll peek under the hood of human and machine perception, playing with ideas at the cutting edge of AI and cognitive science.

Illusions as a Window into the Brain

First, let‘s put on our neuroscience hats and talk about how illusions reveal the inner workings of our visual cortex. Vision feels effortless and instantaneous, but it‘s actually a stunningly complex feat of parallel processing honed by millions of years of evolution.

Our eyes take in raw sensory data – photons hitting the retina. But what we ultimately perceive is a heavily processed model constructed by our brains. Various cortical regions handle specialized tasks like edge detection, motion tracking, color processing, depth estimation, object recognition, etc. The final experience emerges from the outputs of these modules flowing up the visual hierarchy.

This constructive nature of perception is what makes illusions possible. Our brains make assumptions to arrive at a coherent interpretation of messy, incomplete sensory inputs. Most of the time, the heuristics work impressively well – but optical illusions are deliberately crafted to break them.

For example, consider the classic Müller-Lyer illusion:

The two horizontal lines are exactly the same length, yet one looks significantly longer. What‘s going on here? Our brains are using the angles of the fins to estimate depth and length, following rules that normally help us judge the sizes of objects in perspective.

But in this contrived case, those inferences lead us astray. Even if you remember the lines are equal, you can‘t unsee the illusion – the unconscious processing happens regardless of your knowledge. Neuroscience experiments have shown that even in people who consciously know the lines are equal length, their visual cortex still represents them as different sizes. The Müller-Lyer hijacks low-level processing below the level of conscious awareness.

AI and Adversarial Examples

Intriguingly, state-of-the-art AI systems fall for illusions and hacks that are conceptually similar to the Müller-Lyer. In a famous 2013 paper, "Intriguing properties of neural networks", computer vision researchers showed that imperceptible changes to an image could cause a deep neural network to completely misclassify it.

For example, they started with an image that a convolutional neural net correctly identified as a panda with 57% confidence. After adding a tiny amount of carefully constructed noise, invisible to the human eye, the network became 99% convinced the same image was a gibbon!

These so-called "adversarial examples" have remarkable parallels with optical illusions. Both involve tiny, specific perturbations that exploit assumptions and shortcuts in the visual processing pipeline to produce dramatic errors.

In the panda example, the added noise breaks the neural network‘s edge detection and texture recognition, causing it to latch onto spurious features. Very loosely speaking, this is analogous to how the Müller-Lyer fins break our brains‘ length estimation routines.

Since that first striking result, an entire field of AI security research has emerged around adversarial attacks and defenses. Understanding the blind spots and failure modes of neural nets is crucial as we deploy AI systems in high stakes domains.

At the same time, cognitive scientists are exploring these parallels to probe how biological perception works. Some researchers have even evolved brand new "illusions" using genetic algorithms, optimizing images to fool Deep Dream style networks in revealing ways.

The boundaries between studying natural and artificial intelligence are blurring in fascinating ways. Optical illusions turn out to be not just a fun party trick, but a deep research tool illuminating core questions about the nature of mind.

Hacking Perception with Python

Of course, this is a Python tutorial – so let‘s get our hands dirty with some code! I‘ll walk through step-by-step examples of creating classic and cutting-edge illusions using scientific Python libraries.

We‘ll primarily use OpenCV for image loading and filtering, Pillow for drawing shapes and patterns, and NumPy for manipulating pixels and doing numerical work. All the code is available in this Github repo, with a Jupyter notebook for interactive playback.

Our first illusion is a classic: the Hermann grid illusion, where ghostly grey blobs seem to appear and disappear in the intersections of a white grid on a black background.

Here‘s how we generate this with Python:

import numpy as np
import cv2 as cv

h, w = 800, 600 
grid_size = 20
grid = np.zeros((w,h), dtype=np.float32)

for x in range(0, w, grid_size):
    cv.line(grid, (x, 0), (x, w), (1,1,1), 1)

for y in range(0, h, grid_size):  
    cv.line(grid, (0, y), (h, y), (1,1,1), 1)

cv.imshow(‘Hermann Grid‘, grid)
cv.waitKey(0)

The key steps:

1. Create a black background image using a NumPy zeros array
2. Use OpenCV‘s line function to draw white horizontal and vertical lines at regular intervals
3. Display the final grid image

When you stare at the intersections, grey spots seem to flicker in your peripheral vision, but disappear when you look at them directly. This is thought to result from lateral inhibition between visual receptive fields and differences in how our visual system processes foveal (center) vs peripheral information.

Next let‘s generate a motion illusion – the peripheral drift illusion, where a sawtooth luminance gradient appears to rotate:

Here‘s the Python code:

from math import pi, cos
import numpy as np
import cv2 as cv

def sawtooth(x):
    return (x + pi) % (2 * pi) - pi

def illusion(t):
    image = np.zeros((1000, 1000))
    ys = np.arange(1000)

    for x in range(1000):
        xs = x + sawtooth(2*pi*(x/100 + 0.9*t))  
        image[ys, x] = 0.5 + 0.5*cos(2*pi* (xs/30 + 0.01*ys))

    return (255*image).astype(np.uint8)

fps, seconds = 60, 10 
for frame in range(fps*seconds):  
    t = frame / fps
    img = illusion(t)
    cv.imshow(‘Peripheral Drift‘, img)
    if cv.waitKey(1) == 27: 
        break  # esc to quit

This one‘s a bit more involved:

1. Define a sawtooth wave function that generates the drifting gradient
2. Loop over each column of pixels in a blank image
3. Calculate a phase-shifted sawtooth value for each pixel based on its x,y position and the current frame time
4. Set the pixel‘s brightness based on a cosine function of the sawtooth
5. Animate the pattern by advancing the frame time in a loop

When you run this, the sawtooth gradients appear to move, even though it‘s a completely static pattern! The illusory motion results from differences in how your visual system processes fine-grained textures in central vs peripheral regions.

As a final example, let‘s create a color assimilation illusion, where identical gray patches look tinted by surrounding stripes:

Python code:

import numpy as np
import cv2 as cv

w, h = 500, 500
illusion = np.zeros((w, h, 3), dtype=np.float32)

red, green = [255, 0, 0], [0, 255, 0]
gray = [128, 128, 128]

for x in range(5, w-5, 50):
    cv.rectangle(illusion, (x, 100), (x+30, 130), gray, -1)
    cv.rectangle(illusion, (x, 370), (x+30, 400), gray, -1)

for i in range(5):    
    cv.line(illusion, (0,   40*i), (250, 40*i), red, 8) 
    cv.line(illusion, (250, 40*i+20), (500, 40*i+20), green, 8)

cv.imshow(‘Color Assimilation‘, illusion)   
cv.waitKey(0)

Again, we break it down:

1. Make a blank color image
2. Define colors for the stripes and gray patches
3. Draw identical gray rectangles in the top and bottom rows
4. Alternate drawing red and green horizontal stripes

Even though the RGB values of the top and bottom gray patches are the same, the top ones look slightly redder and the bottom slightly greener! This demonstrates color assimilation, where our perception of a region‘s color is biased by surrounding hues.

Evolving New Illusions with AI

These classic illusions are fascinating, but an exciting new frontier is using artificial intelligence to discover entirely new illusions. One compelling approach is to use evolutionary algorithms to "breed" illusory images optimized to fool the human visual system.

The core idea is to define a "fitness function" measuring how effectively an image hacks human perception, then use techniques like genetic algorithms or hillclimbing to search the space of possible images.

For example, we could evolve the Müller-Lyer illusion by defining a fitness function based on the estimated length difference between the two lines. Starting from random combinations of shapes, the evolutionary process would gradually amplify variants that maximize the illusory length difference.

Excitingly, this approach has already yielded novel illusions that researchers are studying to probe the inner workings of human vision. For example, a 2019 paper in the journal Cognition used evolving virtual shapes to discover the "Slant-Tilt illusion", where a twisted 3D volume shape creates a striking mismatch between perceived and actual tilt:

No human designer would have come up with this bizarre, almost alien form – but it perfectly exploits quirks of our 3D shape processing to create a super-stimulating illusion.

Implementing evolutionary searches in Python is outside the scope of this tutorial, but if you‘re curious, there are great libraries like DEAP (Distributed Evolutionary Algorithms in Python) to get started. I could see a fun project evolving illusions using StyleGAN-style convolutional networks as the genotype and a discriminator network scoring the fitness.

More broadly, I think we‘ve only begun to scratch the surface of using AI as a tool for scientifically exploring the nature of mind and intelligent information processing. Both in probing human perception by evolving illusions, and in understanding artificial intelligence by studying its quirks and blind spots.

Optical illusions bridge the worlds of science and art – and with the power of Python and modern AI, we can push the boundaries of both. I hope this article has inspired you to appreciate these mind-bending visuals in a new light, and perhaps try your hand at hacking some perception yourself! The code and ideas here are just a starting point – I‘d love to see what illusions you all discover.