Building a Deep Learning Image Classifier with Keras in R

Deep learning has revolutionized the field of computer vision in recent years, enabling machines to classify images with human-like accuracy. Convolutional neural networks (CNNs) are the workhorse of deep learning for image data, capable of automatically learning hierarchical features from raw pixels. While Python is the go-to language for deep learning, it‘s also possible to build powerful image classifiers using R thanks to the Keras library. In this post, we‘ll walk through the process of building a CNN for image classification in R from start to finish.

Introduction to Image Classification with Deep Learning

Image classification is a core computer vision task that involves assigning a label to an image from a predefined set of categories. This has numerous real-world applications, from organizing personal photo collections to diagnosing medical images to enabling self-driving cars to understand their surroundings. Classical approaches to image classification relied on hand-engineered features, but the paradigm has shifted in favor of deep learning which can automatically learn relevant features from data.

At the heart of deep learning for computer vision are convolutional neural networks (CNNs). CNNs are a specialized neural network architecture designed to operate on grid-like data such as images. They work by learning a hierarchy of features, from simple edges and textures in the early layers to more complex visual concepts in the later layers. This allows CNNs to effectively capture the spatial structure in images that is lost in regular fully-connected neural networks.

Deep Learning in R with Keras

While R has historically been more focused on statistical modeling and data analysis rather than deep learning, this has changed in recent years with the release of powerful deep learning libraries like Keras. Keras is a high-level neural network API that allows you to easily build and train deep learning models. It has interfaces for multiple deep learning backends, including TensorFlow, which powers the R version.

To use Keras in R, you first need to install the keras package from CRAN:

install.packages("keras")

Then load the library:

library(keras)

That‘s it! You now have access to the full functionality of Keras directly within your R environment. This allows you to leverage the strengths of R for tasks like data preprocessing and visualization while still being able to build state-of-the-art deep learning models.

Building an Image Classifier in R

Now let‘s get to the fun part – building an actual image classification model in R using Keras. We‘ll be working with the classic MNIST dataset of handwritten digits. The goal is to train a model that can take in an image of a digit and predict the correct label (0-9).

Loading and Preparing the Data

The first step is to load the MNIST data into R. Conveniently, Keras has a built-in function to access this dataset:

mnist <- dataset_mnist()
x_train <- mnist$train$x
y_train <- mnist$train$y
x_test <- mnist$test$x
y_test <- mnist$test$y

This loads the pre-split training and testing data. The images (x) are 28×28 grayscale pixels, while the labels (y) are integers from 0-9.

Before feeding the images into our model, we need to preprocess them by:

1. Reshaping them to 28x28x1 to represent a single color channel
2. Scaling the pixel values from 0-255 to 0-1
3. Converting the labels to categorical one-hot encoded vectors

We can do this concisely in a few lines of R:

# Reshape and rescale  
x_train <- array_reshape(x_train, c(nrow(x_train), 28, 28, 1)) / 255
x_test <- array_reshape(x_test, c(nrow(x_test), 28, 28, 1)) / 255

# One-hot encode the labels
y_train <- to_categorical(y_train, 10)  
y_test <- to_categorical(y_test, 10)

We‘re now ready to define our CNN!

Defining the Model Architecture

The architecture of a CNN refers to the number and types of layers and how they are connected. There are a few key building blocks:

• Convolutional layers: These apply sliding convolutional filters to the input to produce feature maps. The filters are learned during training.

• Pooling layers: These downsample the feature maps by taking the maximum or average over small windows. This helps the model be invariant to small translations.

• Dense layers: After sufficiently downsampling with convolutional and pooling layers, the final feature maps are flattened and passed through regular fully-connected layers to get class scores.

Here‘s the CNN architecture we‘ll use for MNIST:

model <- keras_model_sequential() %>%
  layer_conv_2d(filters = 32, kernel_size = c(3,3), activation = "relu", input_shape = c(28,28,1)) %>% 
  layer_conv_2d(filters = 64, kernel_size = c(3,3), activation = "relu") %>% 
  layer_max_pooling_2d(pool_size = c(2, 2)) %>% 
  layer_dropout(rate = 0.25) %>% 
  layer_flatten() %>% 
  layer_dense(units = 128, activation = "relu") %>%
  layer_dropout(rate = 0.5) %>% 
  layer_dense(units = 10, activation = "softmax")

This model has two convolutional layers with ReLU activations to learn 32 and 64 filters respectively, followed by a max pooling layer to downsample by a factor of 2. We use dropout layers to regularize the model and prevent overfitting. Finally, the feature maps are flattened and passed through two dense layers, ending in a softmax output over the 10 digit classes.

Compiling and Training the Model

Before training, we need to configure the learning process by specifying the loss function, optimizer and metrics to monitor:

model %>% compile(
  loss = "categorical_crossentropy",
  optimizer = optimizer_rmsprop(),
  metrics = c("accuracy")
)

Since this is a multi-class classification problem, we use categorical cross-entropy as the loss function. The optimizer controls how the model weights are updated based on the loss – here we use RMSprop which is generally a good choice. We‘ll track accuracy as the metric.

We‘re now ready to train the model using the fit() function:

history <- model %>% fit(
  x_train, y_train, 
  epochs = 10, batch_size = 128, 
  validation_split = 0.2
)

We train for 10 epochs (full passes over the training data), using a mini-batch size of 128. We set aside 20% of the data as a validation set to monitor overfitting during training. The history object stores metrics from training.

Evaluating Performance

After training, we can evaluate the model‘s performance on the held-out test set:

results <- model %>% evaluate(x_test, y_test)
results

    loss     acc
0.028513092 0.9901 

We obtain over 99% accuracy on the test set – not bad! We can also plot the training and validation accuracy over time:

plot(history)  

This helps assess whether the model has converged or is starting to overfit.

Making Predictions on New Images

Now for the best part – using our trained model to classify new images! We can do this with the predict() function. Let‘s grab an example image from the test set:

img <- x_test[1,,,,]
dim(img) <- c(1, 28, 28, 1)

We need to add an extra dimension to represent the batch size of 1. Now we can feed it to the model:

preds <- model %>% predict(img)
preds

          0    1    2    3    4    5    6    7    8    9
0 0.9996125 0 0 0 0 0 0 0 0 0.0003877044

The model outputs a probability distribution over the 10 classes. We can take the argmax to get the predicted label:

which.max(preds) - 1

7 

So the model predicts this is an image of the digit 7 (class indices are 0-based). Let‘s plot the image to see if it‘s correct:

plot(as.raster(img[1,,,], max = 1))

Indeed it is! Feel free to load other images and make predictions.

Next Steps

We‘ve seen how to train a simple CNN for image classification using Keras in R from start to finish. There are many ways to improve upon this baseline model:

• Experiment with deeper and wider architectures
• Use techniques like data augmentation and batch normalization
• Optimize the learning rate and other hyperparameters
• Leverage transfer learning by starting with a pre-trained model

I encourage you to play around with the code and test these ideas! The full code for this post along with an interactive R notebook can be found on my Github.

Conclusion

Deep learning is a powerful approach for image classification that has achieved remarkable results. Using high-level APIs like Keras, it‘s easy to start building these models directly in R. We walked through the entire process of loading data, defining a CNN architecture, training the model, and evaluating performance. I hope this gives you a template to start applying deep learning to your own image datasets and problems.

While CNNs can seem like black boxes, there is extensive theory behind how and why they work so well. I recommend the Stanford CS231n course notes as a great resource to dive deeper. Happy deep learning!