Building an Automated Brain Tumor Classifier using Deep Learning

Brain tumors are a serious medical condition that affect millions of people worldwide each year. Early detection and accurate diagnosis are critical for improving patient outcomes. However, manually analyzing brain MRI scans to identify tumors is a time-consuming and challenging task that requires specialized expertise.

In recent years, deep learning techniques have emerged as a powerful tool for automating the analysis of medical images. In particular, convolutional neural networks (CNNs) have achieved state-of-the-art performance on tasks like classifying different types of brain tumors from MRI scans.

In this article, we will walk through the process of building a CNN-based deep learning model to automatically classify brain tumors into different categories. We‘ll use a publicly available dataset of brain MRI images and the popular Keras deep learning library. By the end, you‘ll understand the key components involved in developing these types of AI-powered medical imaging tools.

The Brain Tumor Image Dataset

For this project, we‘ll be using a brain tumor classification dataset that contains 7022 T1-weighted contrast-enhanced MRI images from 233 patients. The images are divided into a training set of 5712 images and a testing set of 1310 images.

The scans cover four main types of brain tumors:

1. Glioma – a tumor that originates in the glial cells of the brain or spine. Gliomas make up about 30% of all brain tumors.

2. Meningioma – a usually benign and slow-growing tumor that forms on the membranes covering the brain and spinal cord. About 20-30% of primary brain tumors are meningiomas.

3. Pituitary adenoma – a typically benign tumor that develops in the pituitary gland at the base of the brain. Pituitary tumors make up around 10% of primary brain tumors.

4. No tumor – these are scans with no detectable tumor present, included to evaluate the model‘s ability to avoid false positives.

The scans have been labeled by expert radiologists to serve as ground truth for training the machine learning model. Each image is 512×512 pixels with pixel intensity values ranging from 0 to 255.

Data Preprocessing

Before training a neural network on the image data, we first need to perform some preprocessing steps. This will help the model learn more effectively from the raw pixel values.

One important technique is data augmentation. This involves randomly transforming the training images on-the-fly to expand the effective size of the dataset. By applying operations like rotations, zooms, flips and shifts to the images, the model is exposed to a wider variety of positions, orientations and scales of the tumors. This helps combat overfitting and improve the generalization of the trained network to new patient scans.

Here we use the ImageDataGenerator class from Keras to set up generators that will read images from disk and apply the specified augmentations:

from tensorflow.keras.preprocessing.image import ImageDataGenerator

train_datagen = ImageDataGenerator(
    rescale=1./255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True,
    vertical_flip=True,
    validation_split=0.2)

test_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
    ‘train‘,
    target_size=(224, 224), 
    batch_size=128,
    class_mode=‘categorical‘,
    subset=‘training‘)

validation_generator = train_datagen.flow_from_directory(
    ‘train‘,
    target_size=(224, 224),
    batch_size=128,
    class_mode=‘categorical‘, 
    subset=‘validation‘)

test_generator = test_datagen.flow_from_directory(
    ‘test‘,
    target_size=(224, 224),
    batch_size=128,
    class_mode=‘categorical‘)

Some key settings:

• rescale=1./255 normalizes the pixel values from 0-255 to 0-1
• shear_range, zoom_range, and horizontal/vertical flips specify the types of random augmentations to apply
• target_size resizes all images to 224×224 for consistency
• validation_split holds out 20% of the training data for validation

This provides us with generators that will yield batches of augmented images and labels that we can directly feed into the model. The class_mode=‘categorical‘ argument tells Keras to automatically convert the string labels into one-hot encoded vectors.

CNN Model Architecture

Now we‘re ready to define the architecture of our convolutional neural network. CNNs have been tremendously successful at image classification tasks due to their ability to automatically learn hierarchical feature representations from raw pixel data.

A typical CNN consists of multiple convolutional layers that apply learned filters to the input, interspersed with pooling layers that reduce the spatial dimensions. This is followed by one or more densely connected layers that perform the final classification.

Here is the Keras code defining our model:

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Conv2D, MaxPool2D, Flatten, Dropout, BatchNormalization

model = Sequential([
    Conv2D(32, kernel_size=(3, 3), activation=‘relu‘, input_shape=(224, 224, 3)),
    MaxPool2D(pool_size=(2, 2)),
    BatchNormalization(),

    Conv2D(64, kernel_size=(3, 3), activation=‘relu‘),
    MaxPool2D(pool_size=(2, 2)),
    BatchNormalization(),

    Conv2D(128, kernel_size=(3, 3), activation=‘relu‘),
    MaxPool2D(pool_size=(2, 2)),
    BatchNormalization(),

    Conv2D(256, kernel_size=(3, 3), activation=‘relu‘),
    MaxPool2D(pool_size=(2, 2)),
    BatchNormalization(),

    Flatten(),
    Dense(256, activation=‘relu‘),
    Dropout(0.5),
    Dense(128, activation=‘relu‘),
    Dropout(0.5),
    Dense(4, activation=‘softmax‘)
])

This network has four convolutional layers with an increasing number of filters (32, 64, 128, 256), each followed by max pooling to halve the dimensions. The BatchNormalization layers help accelerate training.

After flattening the final feature maps, we add two dense layers of size 256 and 128. The dropout layers in between help regularize the model. Finally, the output layer uses the softmax activation to yield a probability distribution over the four tumor classes.

Model Training

With the model architecture defined, we can now compile it with the categorical cross entropy loss function and the Adam optimizer, then kick off training:

model.compile(optimizer=‘adam‘,
              loss=‘categorical_crossentropy‘,
              metrics=[‘accuracy‘])

history = model.fit(
    train_generator,
    steps_per_epoch=100,
    epochs=50,
    validation_data=validation_generator,
    validation_steps=50) 

We train the model for 50 epochs, using the validation set to monitor for overfitting. The fit method returns a history object that we can use to visualize the training progress:

import matplotlib.pyplot as plt

acc = history.history[‘accuracy‘]
val_acc = history.history[‘val_accuracy‘]
loss = history.history[‘loss‘]
val_loss = history.history[‘val_loss‘]

epochs = range(1, len(acc) + 1)

plt.plot(epochs, acc, ‘bo‘, label=‘Training acc‘)
plt.plot(epochs, val_acc, ‘b‘, label=‘Validation acc‘)
plt.title(‘Training and validation accuracy‘)
plt.legend()
plt.figure()

plt.plot(epochs, loss, ‘bo‘, label=‘Training loss‘)
plt.plot(epochs, val_loss, ‘b‘, label=‘Validation loss‘) 
plt.title(‘Training and validation loss‘)
plt.legend()
plt.show()

This plots the accuracy and loss curves so we can verify that the model is converging well without overfitting to the training data.

Model Evaluation

Once we‘re satisfied with the training results, we can evaluate the final model performance on the held-out test set:

test_loss, test_acc = model.evaluate(test_generator)
print(‘Test accuracy:‘, test_acc)

On this dataset, the CNN achieves around 96% classification accuracy on the test images. This demonstrates the potential of deep learning to accurately detect and categorize brain tumors from MRI data.

To get a sense for the model‘s predictions, we can visualize a sample of test images along with the true and predicted labels:

import numpy as np
from sklearn.metrics import confusion_matrix

test_generator.reset()
pred = model.predict(test_generator)
predicted_class_indices=np.argmax(pred,axis=1)

labels = train_generator.class_indices
labels = dict((v,k) for k,v in labels.items())
predictions = [labels[k] for k in predicted_class_indices]

print(confusion_matrix(test_generator.classes, predicted_class_indices))

fig, axes = plt.subplots(nrows=4, ncols=4, figsize=(15, 15))
for i, ax in enumerate(axes.flat):
    ax.imshow(test_generator[i][0][0])
    if test_generator[i][1][0][np.argmax(test_generator[i][1][0])] == predictions[i]:
      color = "green" 
    else:
      color = "red"

    ax.set_title(f"True: {test_generator[i][1][0]}\nPredicted: {predictions[i]}", 
                 color=color)
    ax.axis(‘off‘)

plt.tight_layout()
plt.show()

This gives us a confusion matrix showing the class-wise accuracies, as well as a 4×4 grid of example images with the model‘s predictions annotated (green for correct, red for incorrect).

Applications and Impact

An accurate AI system for brain tumor classification from MRI scans has the potential to make a significant positive impact in clinical settings. Some key benefits:

• Earlier detection – The model could rapidly screen scans and flag potential tumors for expert review as soon as images are acquired, enabling earlier interventions.

• Improved accuracy – Deep learning algorithms can match or even exceed human-level performance, potentially catching subtle cases that may be missed by the human eye.

• Efficiency – Automated analysis allows doctors to focus their time and expertise where it matters most rather than tedious manual examination of scans.

• Accessibility – A software tool deployed via the cloud could enable high-quality diagnosis in regions lacking specialized neuroradiology expertise.

Of course, such AI models are intended to augment rather than replace human doctors. Potential tumors flagged by the system would be confirmed by an expert before proceeding with treatment. Over time, as the technology continues to advance, it could progress to making more detailed characterizations of tumor types, grades, and even genetic subtypes.

Limitations and Future Work

While the trained CNN achieved strong classification performance on this dataset, there are many areas for further development:

• Explainability – In medical contexts, understanding the basis for the AI‘s predictions is crucial. Techniques like class activation mapping or saliency maps could be used to visualize the regions of the scan that most influenced the output.

• Confidence estimation – In addition to the predicted class, having a measure of the model‘s confidence would be valuable. This could be used to prioritize uncertain cases for expert review. Bayesian neural networks are one approach to modeling uncertainty.

• 3D inputs – This model operates on 2D slices, but tumors are 3D structures. Considering multiple adjacent slices or training on full 3D MRI volumes could provide additional context to improve accuracy.

• More data – The 7000 images used here are a good start, but robust clinical deployment would require validation on much larger and more diverse datasets from different institutions, scanners, and patient populations. Continual learning approaches could allow the model to keep adapting to new data over time.

Conclusion

In this article, we walked through the process of building a CNN to automatically classify brain tumors from MRI scans. We started with a labeled dataset, applied preprocessing and augmentation, defined a model architecture, trained it from scratch, and evaluated performance on a test set.

The strong results obtained demonstrate the potential of deep learning to have real-world clinical impact. With further research and development, this technology could help detect brain tumors earlier and more accurately, ultimately improving patient outcomes on a global scale.

At the same time, we must acknowledge that there is still much work to be done to bring this to fruition. In addition to expanding datasets and testing across diverse clinical settings, issues of explainability, confidence estimation, and integration into care delivery workflows will be key.

As deep learning continues to advance, we can expect to see more AI-powered tools transforming medical imaging and other domains. Staying on top of these cutting-edge techniques will be increasingly important for data scientists and developers. I hope this tutorial has given you a practical foundation to start applying CNNs and inspires you to contribute to this impactful area!