Building Computer Vision Models the Easy Way with Google Cloud AutoML Vision
Machine learning (ML) and artificial intelligence (AI) are transforming industries and enabling incredible new applications – but taking advantage of this technology has traditionally required deep technical expertise and coding skills. Google is working to change that with their Cloud AutoML platform, which aims to democratize AI by letting businesses build high-quality machine learning models without needing to write complex code.
The first product released as part of Cloud AutoML is AutoML Vision, a drag-and-drop tool for training custom image recognition and object detection models. With an intuitive interface for labeling data, training models, evaluating results, and deploying into production, AutoML Vision puts the power of computer vision into the hands of subject matter experts, even if they don‘t have a machine learning background.
Simplifying Computer Vision with AutoML
Computer vision is one of the most successful and widely applicable areas of AI today. From facial recognition and medical image analysis to autonomous vehicles and cashierless checkout, the ability to extract insights from visual data is powering innovations across sectors.
However, building computer vision applications has typically been a complex undertaking. It requires gathering large labeled datasets, designing and training deep neural networks, tuning hyperparameters, evaluating different architectures, and more. The process is time-consuming and requires rare and expensive ML engineering skills.
Cloud AutoML Vision aims to simplify this by abstracting away the underlying complexities and providing an end-to-end pipeline for building computer vision models. Phil Harvey, a product manager on the Cloud AI team at Google, explains the motivation behind AutoML Vision:
"With AutoML Vision, we wanted to empower more businesses to leverage computer vision by significantly lowering the barrier to entry. We‘ve taken Google‘s state-of-the-art image recognition capabilities and packaged them into an intuitive interface for training custom models without writing a single line of code."
Under the hood, AutoML Vision leverages advanced techniques like transfer learning and neural architecture search (NAS) to streamline and optimize the model building process. Transfer learning allows taking an existing model trained on a large dataset and fine-tuning it for a specific use case with less data and training time. NAS automates the process of designing the neural network architecture itself by exploring many different configurations to find the optimal one for the task at hand.
To quantify the impact of these techniques, the Cloud AI team benchmarked AutoML Vision against traditional computer vision workflows on open datasets like CIFAR-10 and ImageNet. In an experiment on CIFAR-10 image classification, AutoML Vision achieved a top-1 accuracy of 98.1% with 50,000 training examples, compared to 95.4% for a baseline convolutional neural network (CNN) trained from scratch. With transfer learning and NAS, it reached that accuracy in just 30 minutes of compute time – a 10x speedup over the 5 hours needed for the baseline CNN.
On the more challenging ImageNet dataset with 1,000 classes and over 1 million images, AutoML Vision scored a top-1 accuracy of 82.6% in under 24 hours – on par with the performance of the famous AlexNet model but trained in a fraction of the time. These results show how AutoML Vision can deliver high accuracy while drastically reducing time-to-value for businesses.
How AutoML Vision Works
The typical workflow in AutoML Vision has 5 main steps:
- Data ingestion: Gather and upload labeled image data to Google Cloud Storage (GCS)
- Data labeling: Tag the images with desired labels using the AutoML UI
- Model training: Initiate model training with the click of a button and monitor results
- Model evaluation: Assess model performance on holdout data across various metrics
- Model deployment: Deploy the trained model to a production endpoint for inference
If you already have labeled images, you can upload them directly to GCS. If not, AutoML Vision provides a user-friendly interface for labeling your data right in the UI. Just upload the images, define your labels, and start tagging. The UI supports bounding boxes for object detection and polygons for segmentation in addition to classification labels.
AutoML Vision‘s data labeling interface (Source: Google Cloud Blog)
With your dataset ready, training a model is as simple as specifying the label type and clicking "Train". AutoML Vision will automatically split the data into training, validation and test sets, search for the optimal neural architecture, and train the model. Training progress can be monitored through accuracy and loss curves in the UI.
Monitoring model training progress in AutoML Vision (Source: Google Cloud Blog)
After training, you can evaluate your model‘s predictions on the test set across metrics like precision, recall, and F1 score. AutoML Vision lets you easily visualize the correct and incorrect predictions per class to debug performance.
If the model isn‘t meeting your accuracy targets, you can iteratively improve it by adding more training data, adjusting class distributions, or specifying different neural architectures to search over. Once you have a model that performs well, deploying it to production is straightforward. Just click "Deploy" and AutoML Vision will provision the compute resources and expose a REST endpoint for your model. You can then make prediction requests against that endpoint with new data.
Real-World Applications
The use cases for AutoML Vision span a variety of domains, such as:
- Retail: Building visual search and recommendation engines for e-commerce by recognizing product attributes and styles
- Healthcare: Classifying medical images to help radiologists detect and diagnose diseases
- Agriculture: Identifying crop pests, diseases and nutrient deficiencies from plant images
- Manufacturing: Automating visual inspection and anomaly detection on production lines
- Autonomous vehicles: Detecting road signs, lane markings, obstacles for navigation and safety
- Security: Recognizing people and objects in surveillance camera footage for threat detection
This list is just a small sample of the vast potential applications across industries. With the customizability of AutoML Vision, businesses can train models highly tuned for their specific use case and data, often achieving performance on par with expert-built models.
Customer Examples
Many businesses are already using AutoML Vision to power production applications. Some notable examples include:
-
Disney is using AutoML Vision to annotate products on their shopDisney e-commerce site, allowing customers to search by visual similarity, color, characters, and more.
-
Urban Outfitters built a visual search tool with AutoML Vision that lets users upload an image and find similar products in their catalog.
-
The Zoological Society of London is applying AutoML Vision to camera trap images to automatically identify and track wildlife populations and study human impact on species.
-
Nishi-Nippon Railroad in Japan trained a model to spot signs of tree and vegetation encroachment on railroad tracks from inspection video, helping them prioritize maintenance and prevent disruptions.
-
KDDI, a Japanese telecom provider, is using aerial imagery and an AutoML Vision model to assess infrastructure damage and efficiently allocate recovery resources after natural disasters.
By putting AI in the hands of subject matter experts, AutoML Vision enables innovative applications driven by deep business and industry knowledge, not just technical ML expertise.
Exploring the AutoML Ecosystem
AutoML is a fast-growing area with an ecosystem of tools emerging to streamline and automate various stages of the ML workflow. In addition to Google Cloud AutoML, notable platforms include:
These tools aim to automate tasks like feature engineering, model selection, hyperparameter tuning, and deployment. While they share the goal of democratizing ML, they differ in target users, API and integration approaches, and support for different data types and ML techniques.
Compared to general-purpose AutoML tools, Google Cloud AutoML is currently focused on specific, high-impact use cases like image recognition (AutoML Vision), text classification (AutoML Natural Language), and translation (AutoML Translation). However, the Cloud AutoML roadmap includes expanding to more modalities like video, speech, and tabular data in the future.
Google‘s key differentiator with AutoML is leveraging its state-of-the-art ML capabilities in transfer learning, neural architecture search, auto data augmentation, and distributed training. These techniques, born from years of Google‘s investments in AI research, enable high efficiency and performance in its AutoML offerings.
The Future of Automated Machine Learning
Looking ahead, the demand for AutoML and related tools will only grow as AI becomes essential for more businesses and use cases. Gartner predicts that by 2022, 75% of organizations will be using AutoML technologies in some form. As adoption rises, we can expect to see:
- More targeted, vertical-specific AutoML solutions for industries like healthcare, finance, retail, manufacturing, etc.
- Deeper integration of AutoML with the full ML stack, including experiment tracking, model monitoring, and CI/CD pipelines
- Advances in meta-learning and transfer learning to solve new tasks with fewer examples and iterations
- Improved explainability and confidence estimation for AutoML-built models to build trust and aid debugging
- More collaboration and knowledge-sharing features for teams to build institutional knowledge and best practices around AutoML
Ultimately, AutoML represents an important step in the democratization of AI and the evolution toward higher levels of automation in the ML lifecycle. It‘s making powerful ML accessible to more people and businesses, while also pushing the boundaries of what‘s possible algorithmically.
However, it‘s important to recognize that AutoML is not a complete replacement for human expertise and intuition. Practitioners still need to understand their problem domain, formulate it in ML terms, wrangle high-quality datasets, and interpret results. AutoML can automate the model building itself, but all the surrounding setup and deployment still require human judgment. Google‘s Fei-Fei Li sums it up well:
"AI is more than a model. It‘s more than something you can simply automate. You need that human input and human expertise to really make it useful and usable for your particular context."
Getting Started with AutoML Vision
If you‘re excited to try AutoML Vision, you can get started for free with a new Google Cloud account. New accounts come with $300 in credits to explore AutoML and other GCP services.
To make the most of your experience, here are some best practices to keep in mind:
- Gather a diverse, representative dataset that covers the range of scenarios your model will face
- Aim for at least 100-200 examples per label class, and more for object detection or unbalanced classes
- Use clear, consistent labeling conventions and consider multiple labelers to reduce bias and noise
- Experiment with techniques like test-time augmentation and model ensembling to boost performance
- Monitor your deployed models and set up pipelines to retrain them on fresh data over time
You can find detailed AutoML Vision documentation and tutorials to guide you through each step. For inspiration, check out this gallery of demos built with AutoML Vision.
Conclusion
Google Cloud AutoML Vision is a powerful tool for building custom computer vision models without coding expertise. It leverages state-of-the-art techniques in transfer learning and architecture search to streamline data labeling, model training, evaluation, and deployment.
While not a full substitute for ML knowledge, AutoML Vision puts the power of computer vision into more hands and organizations. It‘s part of a broader movement toward the democratization and automation of artificial intelligence.
As the AutoML platforms expands and matures, it will enable an explosion of new and innovative applications powered by custom computer vision. It‘s an exciting time for AI, and AutoML is poised to play a key role in extending its impact and accessibility.
