Building an Inception Network from Scratch in Python
Deep learning and convolutional neural networks (CNNs) have revolutionized the field of computer vision in recent years, achieving unprecedented performance on tasks like image classification, object detection, and semantic segmentation. One of the most influential CNN architectures to emerge is the Inception network, first introduced by Christian Szegedy and colleagues at Google in 2014. In this article, we‘ll take an in-depth look at the innovations of the Inception architecture and walk through how to implement it from scratch in Python.
Inception Architecture Overview
Prior to Inception, most state-of-the-art CNN architectures, like AlexNet and VGG, relied on stacking convolutional layers deeper and deeper to improve performance. In contrast, the key idea of Inception is to have filters with multiple sizes operate on the same level, covering both local features via smaller convolutions and more abstract features with larger convolutions. This allows the network to recover both fine-grained details and high-level semantics.
The Inception architecture is based on several design principles:
- Avoid representational bottlenecks, especially early in the network
- Higher dimensional representations are easier to process locally within a network
- Spatial aggregation can be done over lower dimensional embeddings without much loss of information
- Balance the width and depth of the network
Based on these principles, a key building block known as the Inception module was developed. The naive form of the Inception module performs convolutions with different filter sizes (1×1, 3×3, 5×5) and a max pooling operation in parallel, concatenating all the outputs together. However, even a small number of 5×5 convolutions can be prohibitively expensive on top of a convolutional layer with a large number of filters.
To address this issue, the dimensionality of the input space is first reduced before the expensive convolutions through 1×1 convolutions. For example, an input with 100 channels can be reduced to 10 channels before applying 5×5 convolutions. Additionally, 1×1 convolutions with a larger number of filters are used to compute reductions before the 3×3 and 5×5 convolutions. Max pooling is still used, but the outputs are then reduced by a 1×1 convolution to keep the volume from exploding. All of these modifications create the final dimension-reduced Inception module.
The full Inception network architecture (also known as GoogLeNet, named after Google) consists of stacks of these Inception modules along with max pooling layers to reduce dimensions. Some other key features of the architecture include:
- 22 layers deep (27 if including the pooling layers)
- Uses global average pooling instead of fully connected layers at the end
- Introduces two auxiliary classifiers connected to intermediate layers during training to combat the vanishing gradient problem (these auxiliary networks are discarded at inference time)
- No use of fully connected layers, saving a large number of parameters
GoogLeNet was the winner of the ILSVRC 2014 image classification challenge, achieving a top-5 error rate of 6.7%, a significant improvement over the 16.4% error of the second place AlexNet from two years prior. It demonstrated how an efficient architecture could achieve high performance with substantially fewer parameters and computational cost.
Implementing Inception in Python
Now let‘s walk through how to implement the Inception v1 architecture in Python using the PyTorch library. The complete code is available on GitHub, but we‘ll break down the key components here.
We start by defining a conv_block function that consists of a convolution followed by batch normalization and a ReLU activation:
class conv_block(nn.Module):
def __init__(self, in_channels, out_channels, **kwargs):
super(conv_block, self).__init__()
self.relu = nn.ReLU()
self.conv = nn.Conv2d(in_channels, out_channels, **kwargs)
self.batchnorm = nn.BatchNorm2d(out_channels)
def forward(self, x):
return self.relu(self.batchnorm(self.conv(x)))Next, we define the Inception module itself, which consists of four parallel branches:
- A 1×1 convolution
- A 1×1 convolution reducing dimensions followed by a 3×3 convolution
- A 1×1 convolution reducing dimensions followed by a 5×5 convolution
- A max pooling operation followed by a 1×1 convolution
class Inception_block(nn.Module):
def __init__(self, in_channels, out_1x1, red_3x3, out_3x3, red_5x5, out_5x5, out_1x1pool):
super(Inception_block, self).__init__()
self.branch1 = conv_block(in_channels, out_1x1, kernel_size=1)
self.branch2 = nn.Sequential(
conv_block(in_channels, red_3x3, kernel_size=1),
conv_block(red_3x3, out_3x3, kernel_size=3, padding=1)
)
self.branch3 = nn.Sequential(
conv_block(in_channels, red_5x5, kernel_size=1),
conv_block(red_5x5, out_5x5, kernel_size=5, padding=2)
)
self.branch4 = nn.Sequential(
nn.MaxPool2d(kernel_size=3, stride=1, padding=1),
conv_block(in_channels, out_1x1pool, kernel_size=1)
)
def forward(self, x):
return torch.cat([self.branch1(x), self.branch2(x), self.branch3(x), self.branch4(x)], 1)With the Inception module defined, we can now build the full Inception network:
class GoogLeNet(nn.Module):
def __init__(self, num_classes=1000):
super(GoogLeNet, self).__init__()
self.conv1 = conv_block(3, 64, kernel_size=7, stride=2, padding=3)
self.maxpool1 = nn.MaxPool2d(3, stride=2, ceil_mode=True)
self.conv2 = conv_block(64, 64, kernel_size=1)
self.conv3 = conv_block(64, 192, kernel_size=3, padding=1)
self.maxpool2 = nn.MaxPool2d(3, stride=2, ceil_mode=True)
self.inception3a = Inception_block(192, 64, 96, 128, 16, 32, 32)
self.inception3b = Inception_block(256, 128, 128, 192, 32, 96, 64)
self.maxpool3 = nn.MaxPool2d(3, stride=2, ceil_mode=True)
self.inception4a = Inception_block(480, 192, 96, 208, 16, 48, 64)
self.inception4b = Inception_block(512, 160, 112, 224, 24, 64, 64)
self.inception4c = Inception_block(512, 128, 128, 256, 24, 64, 64)
self.inception4d = Inception_block(512, 112, 144, 288, 32, 64, 64)
self.inception4e = Inception_block(528, 256, 160, 320, 32, 128, 128)
self.maxpool4 = nn.MaxPool2d(2, stride=2, ceil_mode=True)
self.inception5a = Inception_block(832, 256, 160, 320, 32, 128, 128)
self.inception5b = Inception_block(832, 384, 192, 384, 48, 128, 128)
self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
self.dropout = nn.Dropout(0.2)
self.fc = nn.Linear(1024, num_classes)
def forward(self, x):
# N x 3 x 224 x 224
x = self.conv1(x)
# N x 64 x 112 x 112
x = self.maxpool1(x)
# N x 64 x 56 x 56
x = self.conv2(x)
# N x 64 x 56 x 56
x = self.conv3(x)
# N x 192 x 56 x 56
x = self.maxpool2(x)
# N x 192 x 28 x 28
x = self.inception3a(x)
# N x 256 x 28 x 28
x = self.inception3b(x)
# N x 480 x 28 x 28
x = self.maxpool3(x)
# N x 480 x 14 x 14
x = self.inception4a(x)
# N x 512 x 14 x 14
x = self.inception4b(x)
# N x 512 x 14 x 14
x = self.inception4c(x)
# N x 512 x 14 x 14
x = self.inception4d(x)
# N x 528 x 14 x 14
x = self.inception4e(x)
# N x 832 x 14 x 14
x = self.maxpool4(x)
# N x 832 x 7 x 7
x = self.inception5a(x)
# N x 832 x 7 x 7
x = self.inception5b(x)
# N x 1024 x 7 x 7
x = self.avgpool(x)
# N x 1024 x 1 x 1
x = torch.flatten(x, 1)
# N x 1024
x = self.dropout(x)
x = self.fc(x)
# N x 1000 (num_classes)
return xThe network closely follows the architecture laid out in the original paper, with a few minor changes like using an adaptive average pooling layer at the end instead of a fixed 7×7 size.
One additional component is the auxiliary classifier, which is used during training to inject additional gradient at lower layers and combat the vanishing gradient problem. We attach auxiliary classifiers to the outputs of the inception4a and inception4d modules:
class InceptionAux(nn.Module):
def __init__(self, in_channels, num_classes):
super(InceptionAux, self).__init__()
self.avgpool = nn.AvgPool2d(kernel_size=5, stride=3)
self.conv = conv_block(in_channels, 128, kernel_size=1)
self.fc1 = nn.Linear(2048, 1024)
self.fc2 = nn.Linear(1024, num_classes)
def forward(self, x):
# aux1: N x 512 x 14 x 14, aux2: N x 528 x 14 x 14
x = self.avgpool(x)
# aux1: N x 512 x 4 x 4, aux2: N x 528 x 4 x 4
x = self.conv(x)
# N x 128 x 4 x 4
x = torch.flatten(x, 1)
# N x 2048
x = F.dropout(x, 0.5, training=self.training)
# N x 2048
x = F.relu(self.fc1(x), inplace=True)
# N x 1024
x = F.dropout(x, 0.5, training=self.training)
# N x 1024
x = self.fc2(x)
# N x num_classes
return xThe loss during training is a weighted sum of the auxiliary and final classifier losses:
criterion = nn.CrossEntropyLoss()
loss = criterion(outputs, labels) + 0.3 * (criterion(aux1, labels) + criterion(aux2, labels))During inference, the auxiliary classifiers are not used.
Later Inception Versions
After the success of the original Inception architecture, several refined versions were introduced that incorporated further optimizations:
- Inception v2 and v3 (2015): Replaced 5×5 convolutions with two stacked 3×3 convolutions, used batch normalization, and included factorized 7×7 convolutions.
- Inception v4 and Inception-ResNet (2016): Combined the Inception architecture with residual connections, showing that residual Inception networks outperform similarly expensive Inception networks without residual connections.
These later versions continued to push the state-of-the-art performance on the ImageNet classification challenge.
Conclusion and Applications
The Inception architecture was a milestone in the development of convolutional neural networks, demonstrating how an efficient multi-scale architecture could achieve high accuracy with relatively low computational cost. The innovations it introduced, like dimensionality reduction with 1×1 convolutions and auxiliary classifiers, have been widely adopted and built upon in later architectures.
Inception networks and their variants have been applied to a wide variety of computer vision tasks beyond image classification, including object detection, semantic segmentation, and facial recognition. They continue to be widely used in production systems at Google and other companies.
In this article, we took an in-depth look at the original Inception architecture and walked through a complete implementation in Python and PyTorch. I hope this gives you a solid understanding of this groundbreaking architecture and how to leverage it in your own deep learning computer vision projects. The complete code is available on my GitHub – feel free to experiment with it and adapt it to your own applications!
