Enhancing Ship Classification with CNNs and Transfer Learning
Introduction
Ship classification is an important task in the maritime industry with applications ranging from cargo transport planning to fishing activity monitoring to defense and security. Being able to automatically detect and categorize ships in satellite images enables maritime organizations to efficiently track vessel activity across the globe.
However, ship classification from imagery is a challenging machine learning problem due to several factors:
- High visual similarity between some ship classes
- Large variations in ship appearance due to different sizes, orientations, and lighting conditions
- Presence of clouds, waves, and other objects that can occlude or distort the ship‘s appearance
- Scarcity of large labeled datasets for training deep learning models
In recent years, convolutional neural networks (CNNs) have achieved state-of-the-art results on ship classification tasks, outperforming traditional computer vision techniques. CNNs are particularly well-suited for this problem because of their ability to learn hierarchical features directly from raw pixel data.
Transfer learning with CNNs pre-trained on large datasets like ImageNet has emerged as an effective approach for achieving high ship classification accuracy with limited labeled data. By fine-tuning these pre-trained models, we can build powerful ship classifiers that are robust to real-world variations.
In this post, we‘ll dive deep into the use of CNNs and transfer learning for ship classification. We‘ll cover CNN architectures, training techniques, and practical tips for building production-grade models. By the end, you‘ll have a strong understanding of how to apply these methods to your own ship classification problems.
Convolutional Neural Networks for Ship Classification
At the core of modern ship classification pipelines are convolutional neural networks. CNNs are a class of deep learning models that have proven very effective for computer vision tasks like image classification, object detection, and semantic segmentation.
A CNN consists of three main types of layers stacked together:
-
Convolutional layers: These layers learn local visual features by convolving a set of kernels over the input image. The kernels act as feature detectors, activating when they see a particular pattern in the input. As we stack convolutional layers, the network learns increasingly abstract and semantic features.
-
Pooling layers: Pooling layers downsample the feature maps spatially, typically by taking the maximum activation over a local region. This helps the network learn features that are invariant to small spatial shifts. Pooling also reduces the number of parameters deeper in the network.
-
Fully-connected layers: After the convolutional and pooling layers, the resulting features are flattened and passed through one or more fully-connected layers. These layers learn a non-linear mapping from the features to the output classes.
Over the past decade, many CNN architectures have been proposed for image classification, such as AlexNet [^1], VGGNet [^2], ResNet [^3], and EfficientNet [^4]. These architectures differ in the number and arrangement of layers, as well as the use of techniques like skip connections and depthwise separable convolutions.
[^1]: Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 1097-1105. [^2]: Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. [^3]: He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778). [^4]: Tan, M., & Le, Q. (2019, May). Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning (pp. 6105-6114). PMLR.For ship classification, a common approach is to use a CNN architecture with a moderate number of layers, such as ResNet-50 or MobileNetV2. These models provide a good balance between accuracy and computational efficiency.
The choice of architecture also depends on the characteristics of the satellite imagery, such as the spatial resolution and number of spectral bands. For high-resolution imagery, deeper networks may be necessary to capture fine-grained details. For multi-spectral imagery, we can modify the CNN to take in more than three input channels.
Training CNNs for Ship Classification
To train a CNN for ship classification, we need a labeled dataset of satellite images. Each image should be annotated with a bounding box around the ship and a class label indicating the ship type.
Some of the largest public datasets for ship classification include:
- MASATI (MAritime SATellite Imagery) [^5]: 6,212 satellite images of ships with 7 classes
- HRSC2016 (High Resolution Ship Collections 2016) [^6]: 1,061 high-resolution images of ships with 4 classes
- AirBus Ship Detection Challenge [^7]: 192,556 satellite images, with 81,723 images containing ships
Before training the model, it‘s important to preprocess the images to normalize the pixel values and resize the images to a fixed resolution. We can also apply data augmentation techniques to artificially increase the size of the training set and make the model more robust to variations. Some common augmentations for ship classification include:
- Horizontal and vertical flips
- Rotations
- Crops
- Color jittering
- Gaussian noise
During training, the model parameters are optimized to minimize a loss function, which measures the difference between the model‘s predictions and the true labels. For ship classification, cross-entropy loss is commonly used.
To monitor the model‘s performance during training, we typically measure the accuracy and loss on a held-out validation set. This helps us detect issues like overfitting, where the model performs well on the training data but fails to generalize to new data.
Some best practices for training ship classification CNNs include:
- Using a pre-trained model as the starting point (transfer learning)
- Fine-tuning all layers of the model, not just the last layer
- Using a lower learning rate for the pre-trained layers compared to the new layers
- Monitoring the validation loss and accuracy, and stopping training when the validation performance plateaus
- Saving the model checkpoint with the best validation performance
- Evaluating the final model on a separate test set to get an unbiased estimate of real-world performance
Here‘s an example of fine-tuning a pre-trained MobileNetV2 model for ship classification using TensorFlow and Keras:
from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras.layers import Dense, GlobalAveragePooling2D
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam
# Load the pre-trained MobileNetV2 model
base_model = MobileNetV2(weights=‘imagenet‘, include_top=False, input_shape=(224, 224, 3))
# Freeze the base model layers
for layer in base_model.layers:
layer.trainable = False
# Add new classification layers on top
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(128, activation=‘relu‘)(x)
predictions = Dense(num_classes, activation=‘softmax‘)(x)
# Create the final model
model = Model(inputs=base_model.input, outputs=predictions)
# Compile the model
model.compile(optimizer=Adam(lr=0.0001),
loss=‘categorical_crossentropy‘,
metrics=[‘accuracy‘])
# Train the model on the ship classification dataset
model.fit(train_dataset, epochs=10, validation_data=val_dataset)In this example, we load the MobileNetV2 model pre-trained on ImageNet, and freeze the base layers to prevent them from being fine-tuned. We then add new dense layers on top of the base model to adapt it to the ship classification task. Finally, we compile the model with a low learning rate and categorical cross-entropy loss, and train it on our dataset for 10 epochs.
By leveraging transfer learning, we can train a highly accurate ship classifier with a relatively small dataset. The pre-trained model provides a strong starting point, having already learned generic visual features from ImageNet.
Inference and Deployment
Once we have a trained ship classification model, we can use it to make predictions on new satellite images. This is known as inference.
For real-time inference, it‘s important to consider the computational efficiency of the model. Larger models like ResNet-152 may provide slightly higher accuracy, but at the cost of slower inference times. For applications like real-time ship tracking, a smaller model like MobileNetV2 may be preferable.
We can quantify the inference speed of a model by measuring the number of floating point operations (FLOPs) required for a single forward pass, as well as the actual wall-clock time on the target hardware. The following table compares the accuracy and efficiency of several popular CNN architectures on the MASATI ship classification dataset:
| Model | Parameters (M) | FLOPs (G) | Accuracy (%) | Inference Time (ms) |
|---|---|---|---|---|
| MobileNetV2 | 3.5 | 0.3 | 97.3 | 6 |
| ResNet-50 | 25.6 | 3.8 | 98.2 | 15 |
| EfficientNet-B0 | 5.3 | 0.4 | 98.0 | 8 |
As we can see, MobileNetV2 achieves nearly the same accuracy as ResNet-50 while being much smaller and faster. EfficientNet-B0 provides a good middle ground.
Another important consideration for deployment is the memory usage of the model. On edge devices with limited RAM, we may need to use model compression techniques like quantization and pruning to reduce the memory footprint.
To deploy a ship classification model to production, we can use a serving framework like TensorFlow Serving or Seldon Core. These frameworks provide a RESTful API for making predictions on new data, as well as features like model versioning and monitoring.
Here‘s an example of serving a Keras model with TensorFlow Serving:
# Convert the Keras model to a SavedModel format
model.save(‘ship_classifier‘)
# Start the TensorFlow Serving container
docker run -p 8501:8501 --mount type=bind,source=/path/to/ship_classifier,target=/models/ship_classifier -e MODEL_NAME=ship_classifier -t tensorflow/serving
# Make a prediction request
curl -d ‘{"instances": [{"image": "/path/to/image.jpg"}]}‘ -X POST http://localhost:8501/v1/models/ship_classifier:predictIn this example, we first convert our trained Keras model to the SavedModel format expected by TensorFlow Serving. We then start a TensorFlow Serving container, specifying the path to our model and the model name. Finally, we can make a prediction request to the served model by sending a POST request with the input image.
Conclusion
In this post, we‘ve seen how convolutional neural networks and transfer learning can be used to build highly accurate ship classification models from satellite imagery. By leveraging pre-trained models and fine-tuning them on a labeled dataset, we can achieve state-of-the-art performance with limited training data.
Some key takeaways:
- CNNs are the backbone of modern ship classification pipelines, providing a powerful way to learn hierarchical features from raw images
- Transfer learning with models pre-trained on large datasets like ImageNet can significantly improve accuracy and reduce training time
- Data augmentation and preprocessing are important for making the model robust to real-world variations
- Model architecture should be chosen based on the trade-off between accuracy and inference speed
- Deployment considerations include model quantization, serving infrastructure, and monitoring
Ship classification is just one example of the many applications of CNNs and transfer learning in the maritime domain. Other potential use cases include ship tracking, port activity monitoring, and illegal fishing detection.
As satellite imagery becomes increasingly available and affordable, we can expect to see more organizations adopting these AI techniques to improve their maritime operations. By following best practices and staying up-to-date with the latest research, data scientists and machine learning engineers can build ship classification models that are both accurate and efficient.
