10 Automated Machine Learning Libraries for Supervised Learning in Python (Part 2)

Introduction

Automated machine learning, or autoML for short, is an exciting advancement that aims to make the power of machine learning accessible to non-experts. AutoML uses intelligent automation to perform the time-consuming and repetitive tasks involved in applying machine learning to real-world problems, such as data preprocessing, feature engineering, model selection, and hyperparameter tuning. This allows data scientists to be significantly more productive and enables domain experts to leverage machine learning directly.

While still an emerging field, a variety of autoML libraries have been developed, especially for the popular Python programming language. This two-part article series provides an overview of 10 leading Python autoML libraries, with a focus on supervised learning. Part 1 covered Auto-Sklearn, TPOT, Hyperopt, AutoKeras, and MLJAR. In this article, Part 2, we‘ll dive into five additional state-of-the-art autoML libraries: AutoGluon, H2O, PyCaret, AutoVIML, and LightAutoML. For each library, we‘ll discuss its key features, provide a code example, and weigh some of the pros and cons.

Whether you‘re an experienced data scientist looking to optimize your workflow or a domain expert interested in applying machine learning to your field, read on to see how these cutting-edge autoML tools can help you.

AutoGluon

Developed by Amazon, AutoGluon is a powerful autoML library that trains highly accurate models with as little as three lines of code. It emphasizes ease of use, flexible configuration options, and excellent performance across a range of tasks including tabular, image, and text data.

Some key features of AutoGluon include:

• Support for stacked ensembling, where a high-performance model is created from multiple individual models
• Automatic detection and handling of data types such as numerical, categorical, text, and more
• Intelligent hyperparameter tuning and architecture search
• Distributed training for large datasets

Here‘s an example of using AutoGluon to train a model on a tabular dataset:

!pip install autogluon
from autogluon.tabular import TabularPredictor

train_data = pd.read_csv(‘train.csv‘)
test_data = pd.read_csv(‘test.csv‘)
label_column = ‘target‘

predictor = TabularPredictor(label=label_column).fit(train_data, time_limit=120)  
predictions = predictor.predict(test_data)

In just a few lines, AutoGluon will automatically preprocess the data, find the best model, and generate predictions. The time_limit parameter sets the maximum runtime in seconds to train.

AutoGluon is ideal when you need state-of-the-art performance with minimal code. The tradeoffs are less visibility into the training process and models compared to traditional machine learning. AutoGluon is also a relatively new library and doesn‘t have as large of a community as some other autoML tools.

H2O AutoML

H2O is a leading open-source machine learning platform that includes a powerful autoML module. H2O AutoML provides automatic training and tuning of models including random forests, gradient boosting machines, deep neural networks, stacked ensembles, and more.

Standout capabilities of H2O AutoML are:

• Automatic data preprocessing including handling of missing values, one-hot encoding, and standardization
• Intelligent model selection and stacked ensembles
• Automatic hyperparameter optimization using random search and/or grid search
• Capacity to train on datasets with billions of rows via distributed, in-memory processing

Here‘s how to use H2O AutoML in Python:

!pip install h2o
import h2o
from h2o.automl import H2OAutoML

h2o.init()

train_df = h2o.import_file(‘train.csv‘)
test_df = h2o.import_file(‘test.csv‘)

x = train_df.columns
y = ‘target‘
x.remove(y)

aml = H2OAutoML(max_models=10, seed=1, max_runtime_secs=120)
aml.train(x=x, y=y, training_frame=train_df)

preds = aml.predict(test_df)

This code initializes H2O, imports the data, and then trains an AutoML model. The max_models parameter sets a limit on the number of models to train, while max_runtime_secs specifies the runtime limit in seconds.

H2O AutoML is a good choice for big data tasks thanks to its distributed processing. The H2O Flow GUI also makes it easy to visualize the autoML results. Downsides are that H2O has a steeper learning curve than some other autoML libraries and the models it produces are not easily exportable.

PyCaret

PyCaret is an open-source, low-code autoML library in Python that streamlines the entire machine learning workflow from data preparation to model deployment. It‘s well-suited for citizen data scientists and business analysts in addition to ML practitioners.

PyCaret provides features like:

• Support for a variety of tasks including classification, regression, clustering, anomaly detection, and more
• Automatic data preprocessing and feature engineering
• One-line functions to compare multiple models and tune hyperparameters
• Interactive visualizations and an intuitive API for rapid experimentation
• Option to deploy models via cloud services or generate a standalone pipeline

An example usage of PyCaret for classification:

!pip install pycaret
from pycaret.classification import *

data = pd.read_csv(‘data.csv‘) 
exp1 = setup(data, target = ‘class‘, session_id=123) 

top5 = compare_models(n_select = 5)
tuned_top5 = [tune_model(i) for i in top5]
bagged_top5 = [ensemble_model(i) for i in tuned_top5]

best = automl(optimize = ‘Accuracy‘)
predictions = predict_model(best, data=test_df)

Here PyCaret is used to automatically preprocess the data, compare models, tune the top 5, build ensemble models, and finally select the best model based on accuracy. The resulting predictions are generated with one line of code.

PyCaret is great for rapidly prototyping models and empowering non-technical users to leverage machine learning. However, it does abstract away many of the underlying details, so it may not be ideal for all use cases. The interactive visualizations can also be computationally intensive.

AutoVIML

AutoVIML is an autoML library focused on automatic feature engineering and selection to generate robust, high-performing models. It aims to reduce the time and complexity involved in manual data preprocessing and transformation.

Some of the notable aspects of AutoVIML include:

• Automatic detection and handling of data issues like missing values, outliers, and high cardinality
• Generation of new, highly predictive features via target encoding, numerical combinations, and more
• Optimized feature selection to create simpler, more interpretable models
• Evaluation of multiple models including linear, trees, and neural networks

Here‘s an AutoVIML example for regression:

!pip install autoviml
from autoviml.Auto_ViML import Auto_ViML

y_train = train_df[‘target‘].values
train_df = train_df.drop([‘target‘], axis=1)

model, features, trainm, testm = Auto_ViML(train_df, y_train, test_df, 
            scoring_parameter=‘r2‘, hyper_param=‘GS‘, feature_reduction=True)

predictions = model.predict(test_df)

AutoVIML takes in the training features, targets, and test data. It automatically cleans the data, creates new features, and selects an optimized feature set and model. The resulting model can be used to generate predictions on new data.

The automated feature engineering capabilities of AutoVIML can uncover powerful new signals in the data. It also tends to produce simpler models than other autoML approaches. Potential limitations are longer training times and a lack of built-in distributed computing.

LightAutoML

Created by Sber AI Lab, LightAutoML aims to be a fast, flexible autoML framework for tabular, text, and image data. It offers both automatic training and a modular architecture for customization by experienced users.

LightAutoML boasts features such as:

• Automatic detection of data types and task (binary, multiclass, regression, etc.)
• Support for popular ML libraries including LightGBM, CatBoost, linear models, and Transformers
• Automatic hyperparameter tuning via successive halving
• Powerful text and image modeling capabilities
• Detailed experiment tracking and logging

An example of binary classification with LightAutoML:

!pip install -U lightautoml
from lightautoml.automl.presets.tabular_presets import TabularAutoML
from lightautoml.tasks import Task

automl = TabularAutoML(task = Task(‘binary‘),
                       timeout = 600, 
                       gpu_ids = ‘0‘,
                       general_params = {‘use_algos‘: [[‘linear_l2‘, ‘lgb‘]]})

oof_predictions = automl.fit_predict(train_df, roles = {‘target‘: ‘target‘})
test_predictions = automl.predict(test_df)

This code defines a binary classification task, sets a 10-minute timeout, and specifies the use of GPU 0. It also limits the model types to linear models and LightGBM. The automl object is fit on the training data and used to generate predictions.

LightAutoML is built for speed and has advanced capabilities for text and image data. The ability to customize the autoML with different models and architectures is also valuable for expert users. As a newer library, the documentation is still being developed, which could be a drawback for some.

How to Choose an AutoML Library

With many robust autoML options available in Python, it can be hard to know which one to use. Here are a few key factors to consider:

• Ease of use – If you‘re just getting started with ML, a more abstracted library like PyCaret may be a good choice. Experienced practitioners may prefer a more configurable framework like Auto-Sklearn or LightAutoML.
• Performance – For top model accuracy, look to libraries like AutoGluon and H2O AutoML that leverage advanced techniques like stacked ensembling.
• Scalability – If you‘re working with very large datasets, choose a library with distributed computing capabilities such as H2O or MLJAR.
• Specific data types – Some libraries are designed for certain data types, such as AutoKeras for image data or LightAutoML for text. Pick one that matches your data.
• Customizability – Libraries like TPOT and Auto-Sklearn are built on top of scikit-learn and are easy to customize. Others are more of a "black box".

Automated Machine Learning in 2023 and Beyond

AutoML has made significant strides in recent years thanks to advances in meta-learning, neural architecture search, hyperparameter optimization, and other techniques. Cutting-edge research is pushing the boundaries of what‘s possible.

For example, Google Brain recently introduced a new autoML system using massive neural networks with over one trillion parameters. The models are pre-trained on a huge corpus of data and then fine-tuned for specific tasks, resulting in state-of-the-art performance across a range of benchmarks. This points to a future where foundation models are used as a starting point for autoML.

Other promising areas of autoML research include faster neural architecture search, meta-learning to warm-start optimization, and autoML for tasks like time series, anomaly detection, and more. As autoML systems become more robust, they may be able to handle the entire data pipeline from cleaning to deployment.

It‘s an exciting time for automated machine learning, with libraries becoming more powerful and accessible every day. While autoML is not a replacement for human intuition and domain knowledge, it has the potential to significantly augment and scale the capabilities of data scientists and ML engineers. By understanding and leveraging autoML, practitioners can build better models faster and organizations can realize the value of machine learning more quickly. The future is bright for autoML—and this is just the beginning.