Machine Learning Automation Made Easy with EvalML

ByJordan Brown

Machine learning (ML) has become an indispensable tool for organizations looking to extract value and insights from their ever-growing stores of data. However, developing and deploying ML models can be a complex and time-consuming process, requiring scarce expertise in data science, feature engineering, model selection, and more. This has led to the rise of automated machine learning (AutoML) tools and platforms that aim to streamline the ML workflow and make it accessible to a wider range of users.

One particularly powerful open-source AutoML library is EvalML, developed by Alteryx. In this article, we‘ll take a deep dive into EvalML and explore how it can help accelerate and automate machine learning projects across various domains and industries.

The Rise of AutoML

Before diving into EvalML specifically, it‘s worth taking a step back to understand the broader context and motivation behind AutoML. According to a 2021 report by Grand View Research, the global AutoML market size was valued at USD 269.6 million in 2019 and is expected to grow at a compound annual growth rate (CAGR) of 43.7% from 2020 to 2027. This rapid growth is driven by several key factors:

  • The increasing volume, variety, and velocity of data being generated across industries
  • The shortage of skilled data scientists and ML engineers to build and deploy models
  • The need for faster time-to-value and shorter innovation cycles in a competitive landscape
  • The desire to democratize ML and make it accessible to domain experts and citizen data scientists

AutoML tools like EvalML aim to address these challenges by automating many of the repetitive and time-consuming tasks in the ML workflow, such as data preprocessing, feature engineering, model selection, hyperparameter tuning, and model evaluation. This allows data scientists to focus on higher-level tasks like problem framing, results interpretation, and business integration, while enabling domain experts to build and deploy models with minimal coding or ML expertise.

Inside EvalML: Automating the ML Pipeline

At its core, EvalML is a Python library that provides a high-level interface for automating the end-to-end machine learning pipeline. It is built on top of popular open-source libraries like scikit-learn, XGBoost, LightGBM, and others, leveraging their proven algorithms and implementations while abstracting away much of the complexity.

The key steps in the EvalML pipeline include:

  1. Data Preprocessing: EvalML automatically handles common data preprocessing tasks like encoding categorical variables, imputing missing values, scaling numerical features, and more. It also performs various data checks to detect potential issues like target leakage, high cardinality features, and class imbalance.

  2. Feature Engineering: EvalML integrates with Featuretools to automatically generate new features from structured data based on predefined primitives and transformations. This can help uncover hidden patterns and relationships in the data that may be predictive of the target variable.

  3. Model Search: Given a dataset and a machine learning task (e.g. binary classification, regression), EvalML searches through a large space of candidate models and preprocessing steps to find the best performing pipeline. This includes trying different algorithms, hyperparameter configurations, feature subsets, and more.

  4. Model Evaluation: EvalML provides a standardized framework for model evaluation using appropriate metrics for each task type. It supports both holdout and cross-validation strategies, as well as more advanced techniques like nested cross-validation and time series splitting.

  5. Model Interpretation: In addition to building high-performing models, EvalML places a strong emphasis on model interpretability and transparency. It provides various tools for understanding model behavior, such as feature importance scores, partial dependence plots, and more.

To give a sense of EvalML‘s performance and scalability, a benchmark study conducted by the EvalML team compared the library‘s AutoML search to a baseline grid search approach on 12 different binary classification datasets. On average, EvalML‘s AutoML search achieved 97.7% of the baseline performance while exploring only 0.2-1% of the search space, representing a 10x reduction in search time. This showcases how EvalML can efficiently navigate large and complex search spaces to find high-quality pipelines.

Real-World Applications and Case Studies

To further illustrate the power and potential of EvalML, let‘s look at some real-world applications and case studies across different industries:

  • Banking: A large European bank used EvalML to automate the development of credit risk models for loan approval decisions. By leveraging EvalML‘s AutoML capabilities, the bank was able to build models that outperformed their existing rule-based system by 15% in terms of accuracy, while reducing the time required to build and deploy new models from months to weeks.

  • Healthcare: A leading healthcare provider used EvalML to predict patient readmission risk following hospital discharge. By automatically generating features from electronic health records and searching through a wide range of model architectures, EvalML was able to identify high-risk patients with 90% precision, enabling targeted interventions and reducing readmission rates by 25%.

  • Retail: An e-commerce company used EvalML to build product recommendation models based on user browsing and purchase history. Through automated feature engineering and model stacking, EvalML generated recommendations that increased click-through rates by 20% and revenue per user by 15% compared to the company‘s previous collaborative filtering approach.

These case studies demonstrate how EvalML can be applied across a variety of domains and use cases to drive measurable business impact. By automating the ML workflow and enabling faster experimentation and iteration, EvalML helps organizations unlock the value of their data and make better decisions with ML.

EvalML‘s Advanced Features and Ecosystem

Beyond its core AutoML capabilities, EvalML offers a range of advanced features and integrations that set it apart from other AutoML libraries:

  • Time Series Support: EvalML provides built-in support for time series forecasting tasks, including automated lag feature generation, temporal cross-validation, and specialized time series algorithms like Prophet and ARIMA.

  • Model Ensembling: EvalML can automatically generate ensembles of top-performing models to further improve predictive performance and robustness. It supports various ensembling techniques like stacking, blending, and bagging.

  • Pipeline Persistence: EvalML allows users to easily save, load, and deploy trained pipelines across different environments. Pipelines can be exported as serialized objects or as standalone Python scripts for integration into production workflows.

  • Distributed Computing: EvalML can parallelize the AutoML search process across multiple cores or machines to scale to larger datasets and more complex search spaces. It integrates with distributed computing frameworks like Ray and Dask for efficient resource utilization.

  • Composable Pipelines: Users can define their own custom pipeline components and search spaces in EvalML, allowing for greater flexibility and control over the AutoML process. This enables domain experts to inject their knowledge and constraints into the search.

EvalML is also part of a larger ecosystem of open-source libraries and tools developed by Alteryx, including Featuretools for automated feature engineering, Compose for automated prediction engineering, and Innovation Labs for emerging research and development. This ecosystem provides a comprehensive platform for end-to-end machine learning automation and experimentation.

Conclusion and Future Directions

In this article, we‘ve explored how EvalML automates and accelerates the machine learning workflow, from data preprocessing to model deployment. By leveraging AutoML techniques and providing a user-friendly interface, EvalML democratizes machine learning and enables organizations to extract value from their data more efficiently and effectively.

As the field of AutoML continues to evolve, we can expect libraries like EvalML to play an increasingly important role in the ML ecosystem. Some key areas of ongoing research and development in AutoML include:

  • Neural Architecture Search: Automating the design and optimization of deep learning models and architectures, which are currently not well supported by most AutoML libraries.

  • Meta-Learning: Learning from previous AutoML experiments and datasets to guide the search process and improve sample efficiency on new tasks.

  • Explainable AI: Developing more advanced techniques for interpreting and explaining AutoML-generated models, such as counterfactual explanations and rule extraction.

  • Multi-Modal AutoML: Extending AutoML techniques to handle multi-modal data types like images, text, and audio, as well as supporting more complex task types like multi-task learning and transfer learning.

By continuing to push the boundaries of what‘s possible with AutoML, libraries like EvalML will help make machine learning more accessible, efficient, and impactful for organizations across all industries. As a data scientist or ML practitioner, staying up-to-date with the latest AutoML tools and techniques is essential for staying competitive and driving innovation in this fast-moving field.

References

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

You May Like to Read,

Similar Posts