Automated Deep Learning with AutoKeras: A Guide to Classification and Regression

Deep learning has transformed the field of machine learning in recent years, enabling unprecedented performance on complex tasks like image classification, natural language processing, and predictive analytics. However, building deep neural networks remains a challenging and time-consuming process that requires significant expertise.

This is where automated machine learning (AutoML) comes in. AutoML aims to make deep learning more accessible by automating many of the manual steps involved in designing neural architectures, tuning hyperparameters, and training models. One of the leading open-source libraries for AutoML is AutoKeras, which enables beginners and experts alike to quickly build state-of-the-art deep learning models for tasks like classification and regression.

In this guide, we‘ll take a deep dive into AutoKeras and show you how to use it to tackle real-world machine learning problems. Whether you‘re new to deep learning or an experienced practitioner looking to streamline your workflow, AutoKeras is a powerful tool to have in your arsenal. Let‘s get started!

What is AutoKeras?

AutoKeras is an open-source library for automated machine learning developed by the DATA Lab at Texas A&M University. It provides a high-level, Keras-based API for automatically finding the best deep learning architecture and hyperparameters for a given dataset and task.

Some key features of AutoKeras include:

• Automated neural architecture search to find optimal models
• Support for structured data, images, and text
• Easy-to-use APIs for classification and regression problems
• Compatibility with GPUs for accelerated training
• Integration with the Keras and TensorFlow ecosystems

AutoKeras builds upon the popular Keras deep learning framework, so if you‘re already familiar with building models in Keras, you‘ll feel right at home with AutoKeras. The key difference is that instead of manually specifying the layers and hyperparameters in your models, you can let AutoKeras automatically discover the optimal architecture for you.

Under the hood, AutoKeras uses a technique called Neural Architecture Search (NAS) to efficiently explore the space of possible models. It employs a "Greedy Strategy" that starts with a small initial model and gradually grows the architecture by adding new layers. At each step, it selects the most promising candidate model based on performance on a validation set. This allows it to converge on high-quality models much faster than techniques that search the entire space from scratch.

Setting up AutoKeras

Before we dive into some examples, let‘s go over how to set up AutoKeras on your local machine. The easiest way is using the pip package manager:

pip install autokeras

This will install AutoKeras and its dependencies, including TensorFlow. Note that you‘ll need Python 3.5 or later.

If you have a GPU available, it‘s strongly recommended to install the GPU version of TensorFlow to drastically speed up training:

pip install tensorflow-gpu 

You‘ll also need to install the appropriate NVIDIA drivers and CUDA toolkit – see the TensorFlow docs for details.

Example 1: Classification with AutoKeras

To see AutoKeras in action, let‘s walk through an example of building an image classifier on the classic MNIST handwritten digits dataset. We‘ll use AutoKeras to automatically find a deep learning model that can recognize digits with high accuracy.

First, let‘s load the dataset using tf.keras.datasets:

from tensorflow.keras.datasets import mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()

This loads 60,000 28×28 grayscale training images and 10,000 test images, each with a corresponding label from 0 to 9.

Next, let‘s initialize an AutoKeras ImageClassifier with some key parameters:

from autokeras import ImageClassifier

clf = ImageClassifier(max_trials=3, seed=42)

Here we‘re specifying to run a maximum of 3 trials (i.e. 3 different candidate models), and setting a random seed for reproducibility. We can now call fit() on the classifier to begin the search:

clf.fit(x_train, y_train, epochs=10)

AutoKeras will now automatically search for the best model architecture and hyperparameters, training each candidate for 10 epochs. Throughout this process, it will output the current best model and its validation accuracy.

After the search is complete, we can evaluate the final model on our test set:

print(clf.evaluate(x_test, y_test))

On our MNIST example, this yields a stellar accuracy of over 99%! We can also inspect the final model:

model = clf.export_model()
model.summary()

This shows the detailed architecture that was automatically discovered, with convolutional, pooling, and dense layers.

Finally, we can use the trained model to make predictions on new images:

predictions = clf.predict(x_test[:10])
print(predictions)  

We see that AutoKeras was able to find a highly accurate classifier for this task with minimal code and no manual tuning required!

Example 2: Regression with AutoKeras

In addition to classification, AutoKeras supports automated model search for regression problems, where the goal is to predict a continuous target variable.

As an example, let‘s use the built-in California housing prices dataset:

from sklearn.datasets import fetch_california_housing
import numpy as np

housing = fetch_california_housing()
x_train, x_test, y_train, y_test = train_test_split(housing.data, housing.target, test_size=0.2, random_state=42)

We can initialize a StructuredDataRegressor, which is designed for datasets with numerical and categorical features:

from autokeras import StructuredDataRegressor

reg = StructuredDataRegressor(max_trials=5, seed=42)
reg.fit(x_train, y_train, epochs=50)

After running for 50 epochs across 5 trials, the searcher will return the best model it found. We can evaluate it on the held-out test set:

mse, mae = reg.evaluate(x_test, y_test)
print(f"Mean squared error: {mse:.2f}, mean absolute error: {mae:.2f}")

This shows strong performance, with a mean absolute error of around $50,000 on this dataset. We can also visualize the predictions vs actual prices:

import matplotlib.pyplot as plt

y_pred = reg.predict(x_test)
plt.scatter(y_test, y_pred)
plt.xlabel("Actual Price")
plt.ylabel("Predicted Price")
plt.title("California Housing Prices")
plt.show()

The results demonstrate the power of AutoKeras to quickly build high-quality models for both classification and regression tasks with very little code.

Tips and Best Practices

While AutoKeras automates much of the model building process, there are still a few key levers you can use to guide the search and improve results:

1. Adjust max_trials and epochs – The max_trials parameter controls how many different architectures are considered, while epochs sets the training budget for each model. Increasing these will give AutoKeras more opportunities to find better models, but also increases runtime.

2. Tune the validation split – AutoKeras automatically holds out a portion of your training data to evaluate candidate models. You can adjust this split by passing a validation_split or validation_data argument to fit().

3. Specify the evaluation metric – By default AutoKeras uses accuracy for classification and MSE for regression, but you can specify your own metric to optimize, e.g. f1 score or RMSLE, using the metrics argument.

4. Limit the search space – If you have some prior knowledge about what architectures work well for your task, you can constrain AutoKeras‘ search space by passing in a custom Project or Node.

5. Perform your own preprocessing – While AutoKeras will automatically scale numerical features and encode categoricals, it can still be beneficial to do your own feature engineering, dimensionality reduction, etc. before passing data to fit().

When to use AutoKeras

AutoKeras and AutoML in general can be incredibly useful for quickly building baseline models, prototyping different architectures, and automating repetitive tasks. However, it‘s not a complete replacement for human expertise and intuition.

The key tradeoff is generality vs specificity – because AutoML methods are designed to work across a wide range of datasets and problems, they may not always find the absolute best architecture for a particular task. Hand-designed models that incorporate domain knowledge can still outperform AutoML in many cases.

Another potential downside is computational cost. Neural architecture search can be expensive, especially if you‘re working with very large datasets or limited compute resources. In these cases, you may be better off manually tuning a smaller model.

That said, for many common machine learning tasks, AutoKeras can get you 90% of the way there with very little effort. It‘s a great way to quickly establish a strong baseline before investing time in custom model design.

The Future of AutoML

As machine learning becomes increasingly crucial across industries, automated tools like AutoKeras will play a key role in making the technology more accessible and scalable.

Some exciting frontiers in AutoML research include:

• Adaptive search algorithms that learn from previous experiments to speed up optimization
• Transfer learning methods that leverage pretrained models to warm-start the search
• Multitask and lifelong learning approaches that find models to solve many related problems
• Explainable AutoML systems that provide visibility into the architectures they discover

As these techniques continue to evolve, AutoML may soon be able to match or exceed human experts in model design. For now though, the most promising approach is a collaboration between automated tools and human intuition.

Conclusion

In this guide, we‘ve seen how AutoKeras makes it easy to automatically build state-of-the-art deep learning models for classification and regression, without giving up the flexibility and customization of Keras.

Whether you‘re a machine learning beginner looking to get started quickly, or an experienced practitioner looking to automate repetitive tasks, AutoKeras is a powerful tool to add to your toolkit.

The key takeaways from this article are:

• AutoKeras uses neural architecture search to automatically find optimal models for your dataset
• It provides simple, Keras-based APIs for classification and regression with structured data, images, and text
• With a few lines of code, you can often build models that rival hand-tuned architectures
• AutoKeras is not a complete replacement for human expertise, but can accelerate many common machine learning tasks
• AutoML technology is advancing rapidly and poised to democratize AI for many more applications

To learn more about AutoKeras, check out the official documentation and tutorials. The source code is also available on GitHub.

Happy automated modeling!