Plant Seedlings Classification Using CNN with Python Code

Being able to automatically recognize and classify plant seedling species from images has many useful applications, especially in agriculture and farming. For example, a computer vision system on an agricultural robot or drone could identify weeds in a field and precisely spray herbicide only on the unwanted plants while leaving crops untouched. Plant ecologists and botanists could also use such a system to automatically identify and map distributions of plant species over a wide area.

In recent years, deep learning approaches have achieved state-of-the-art results on challenging image classification tasks. In particular, convolutional neural networks (CNNs) have proven very effective at learning hierarchical features from raw image pixels and mapping them to output class labels. A typical CNN architecture consists of several convolutional layers that learn local visual features, pooling layers that reduce spatial dimensions, and fully-connected layers that learn a final mapping to the output classes.

In this tutorial, we‘ll walk through a Python code example of how to build an image classifier for 12 plant seedling species using the Keras deep learning library. We‘ll be using a dataset of 5,539 images from the Plant Seedlings Classification competition on Kaggle. Here are the key steps:

1. Import Libraries

First we‘ll import all the libraries needed for loading data, preprocessing, and building the CNN model:

import numpy as np 
import pandas as pd
import matplotlib.pyplot as plt
import cv2 
from glob import glob
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, BatchNormalization
from keras.preprocessing.image import ImageDataGenerator
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix

2. Load and Preprocess Data

Next we‘ll load the plant seedling images from files using OpenCV and resize them to a fixed size of 70×70 pixels:

path = ‘plant-seedlings-dataset/train/‘
image_files = glob(path + ‘*.png‘)
images = []
labels = []

for filename in image_files:
    image = cv2.imread(filename)
    image = cv2.resize(image, (70,70)) 
    images.append(image)
    labels.append(filename.split(‘/‘)[-2])

images = np.array(images)    

To remove background noise from the images, we can convert to the HSV color space, define a range of green color values, and create a mask to extract just the green plant regions:

masked_images = []

for image in images:
    hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
    lower_green = (25, 40, 50)
    upper_green = (75, 255, 255)
    mask = cv2.inRange(hsv, lower_green, upper_green)
    masked = cv2.bitwise_and(image, image, mask=mask)  
    masked_images.append(masked)

masked_images = np.array(masked_images)

Before training the model, we need to convert the text species labels to numeric values and one-hot encode them:

label_encoder = LabelEncoder()
labels = label_encoder.fit_transform(labels)
labels = to_categorical(labels)

Finally, we‘ll split the images and labels into training and test sets and scale the pixel values to the range [0,1]:

x_train, x_test, y_train, y_test = train_test_split(masked_images, labels, 
                                                    test_size=0.2, random_state=42)
x_train = x_train / 255.0
x_test = x_test / 255.0                                                 

3. Define CNN Model

Now we‘ll define a CNN architecture with 4 convolutional-pooling layer pairs followed by 3 fully-connected layers:

model = Sequential()

model.add(Conv2D(32, (3,3), activation=‘relu‘, input_shape=(70, 70, 3))) 
model.add(MaxPooling2D(2,2))
model.add(BatchNormalization())

model.add(Conv2D(64, (3,3), activation=‘relu‘))
model.add(MaxPooling2D(2,2))
model.add(BatchNormalization())

model.add(Conv2D(128, (3,3), activation=‘relu‘)) 
model.add(MaxPooling2D(2,2))
model.add(BatchNormalization())

model.add(Conv2D(128, (3,3), activation=‘relu‘))
model.add(MaxPooling2D(2,2))
model.add(BatchNormalization())

model.add(Flatten())

model.add(Dense(512, activation=‘relu‘))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Dense(256, activation=‘relu‘)) 
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Dense(12, activation=‘softmax‘))

model.compile(loss=‘categorical_crossentropy‘,
              optimizer=‘adam‘,
              metrics=[‘accuracy‘])

This model uses ReLU activation in the convolutional and dense layers, batch normalization for stability, and dropout regularization to prevent overfitting. The final layer outputs class probabilities using the softmax function.

4. Train Model

To train the model, we‘ll use the fit() function and pass it the training data. To further prevent overfitting, we‘ll generate augmented training images on the fly using an ImageDataGenerator that randomly flips, rotates, and shifts the images:

datagen = ImageDataGenerator(rotation_range=180, 
                             width_shift_range=0.1,
                             height_shift_range=0.1,
                             horizontal_flip=True, 
                             vertical_flip=True)

checkpointer = ModelCheckpoint(‘best_model.h5‘, save_best_only=True)

model.fit(datagen.flow(x_train, y_train, batch_size=32),
          steps_per_epoch=len(x_train)/32,
          epochs=50,
          validation_data=(x_test, y_test),
          callbacks=[checkpointer])

We‘re using a ModelCheckpoint callback to save the model weights for the epoch that achieves the best validation accuracy. Training for 50 epochs with a batch size of 32 takes about 15 minutes on a typical GPU.

5. Evaluate Model

After training, we can evaluate the best saved model on the test set and plot a confusion matrix to see where it makes mistakes:

model.load_weights(‘best_model.h5‘)

y_pred = np.argmax(model.predict(x_test), axis=-1)
y_test = np.argmax(y_test, axis=-1)
cm = confusion_matrix(y_test, y_pred)

plt.figure(figsize=(12,12))
plt.imshow(cm, interpolation=‘nearest‘, cmap=plt.cm.Purples)
plt.show()

On the Kaggle test set of 794 images, this model achieves 93.3% accuracy. The confusion matrix shows that it sometimes confuses similar looking species like Scentless Mayweed and Common Chickweed.

6. Get Predictions

Finally, to use the trained model to classify new seedling images, we can load the model weights and call predict() on an array of preprocessed images:

model.load_weights(‘best_model.h5‘)

test_files = glob(‘plant-seedlings-dataset/test/*.png‘) 
test_images = []

for filename in test_files:
    image = cv2.imread(filename)
    image = cv2.resize(image, (70,70))
    hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)  
    mask = cv2.inRange(hsv, (25,40,50), (75,255,255))
    masked = cv2.bitwise_and(image, image, mask=mask)
    test_images.append(masked)

test_images = np.array(test_images) / 255.0
test_preds = np.argmax(model.predict(test_images), axis=-1)

predicted_species = label_encoder.inverse_transform(test_preds)

Next Steps

There are a number of ways we could potentially improve this plant seedling classifier:

• Gather more training data, especially for species with fewer images
• Systematically tune the CNN architecture and hyperparameters
• Try other model architectures like ResNet and DenseNet
• Use transfer learning to initialize weights from models pretrained on ImageNet
• Combine model predictions with other information like geographic location and time of year

In general, some best practices to keep in mind when building image classifiers with deep learning:

• Make sure your training data is high quality and representative of the problem
• Preprocess images consistently and experiment with different techniques
• Start with a proven model architecture and initialize weights from a pretrained model if possible
• Visualize model performance with confusion matrices and use that to guide improvements
• Deploy your model to get feedback and monitor performance on real-world data

Hopefully this tutorial gives you a good starting point for using CNNs to tackle your own plant recognition and image classification challenges! Let me know if you have any other questions.