# How Open Source Software Powers Modern MLOps - Canonical: https://33rdsquare.com/how-open-source-beats-proprietary-software-helps-for-mlops/ - Published: 2024-09-03 - Author: Jordan Brown - Categories: [Artificial Intelligence & Machine Learning & ChatGPT](https://33rdsquare.com/category/tech/ai/) --- The widespread adoption of machine learning in recent years has given rise to the practice of MLOps – applying DevOps principles to help organizations develop, deploy and maintain ML applications efficiently and reliably. A key factor fueling the growth of MLOps is the availability of powerful open source software. From model development to deployment and monitoring, open source tools have become the foundation of most modern ML infrastructures. In this post, we‘ll examine how open source software provides critical advantages over proprietary alternatives when it comes to enabling MLOps. We‘ll look at the core benefits of open source, the key tools powering MLOps today, and why controlling your own ML platform is so important for serious practitioners. ## The Power of Open Source Let‘s start with the fundamental advantages of open source software. While people are sometimes skeptical of tools that are given away for free, the open source model has proven to drive the creation of software that is often higher quality and more capable than commercial options. There are a few key reasons for this: First, open source projects tend to be created by practitioners to solve real-world problems they face. The developers have an inherent incentive to make the software as useful and robust as possible. By contrast, commercial software is often driven more by sales and marketing priorities than pure engineering considerations. "Engineers at companies like Google and Facebook rely heavily on open source infrastructure, and they contribute back to those projects because they directly benefit from their improvement," explains Hamel Husain, Staff Machine Learning Engineer at GitHub. Second, popular open source projects benefit from having many more eyeballs on the code compared to proprietary software developed behind closed doors. With a large community of users and contributors, bugs and security holes tend to be identified and fixed very quickly. Useful new features can come from anyone who puts in the effort to build them. A 2020 analysis by Trend Micro found that 49% of security bugs in commercial software remained unfixed after 180 days compared to only 21% in open source software. Third, open source projects must live or die on their own merits. With a low barrier to entry and no "vendor lock-in", users can easily switch to a different solution if a tool fails to meet their needs. This forces open source software to continuously innovate and drive the state of the art forward. "Good ideas come from everywhere, not just the team at your company," says Peter Wang, CEO of Anaconda. "Empowering your teams to build upon each other‘s open source work accelerates progress for everyone." These factors help explain why so much critical infrastructure is now built on open source software – everything from operating systems to web servers to big data platforms. Even the largest tech companies rely heavily on open source: - 72% of GitHub‘s Fortune 100 customers use open source software - Over 85% of companies in the S&P 500 use open source software - 90% of IT leaders use open source for "non-differentiating" software This reliance on open source is especially pronounced when it comes to machine learning and MLOps. A 2021 survey by Anaconda found that 85% of data science professionals prefer to use open source software for ML/AI development. ## Why Open Source is Critical for MLOps The advantages of open source are particularly important in an emerging field like MLOps where best practices are still being established and tooling needs are rapidly evolving. ML teams need a high degree of flexibility to experiment, customize workflows, and adapt to changing requirements. Proprietary software that forces you into a predefined way of working can be very limiting. Interoperability is also crucial since MLOps sits at the intersection of data engineering, ML engineering, and DevOps. Tools need to work well with a variety of systems and be able to fit into an organization‘s existing infrastructure. Open source software is typically built around open standards and APIs that make integration much easier than walled-off proprietary platforms. For example, the Open Neural Network Exchange (ONNX) format makes it possible to train a model using one framework like PyTorch and then deploy it using another system like ML.NET. > "Machine learning models are meant to be integrated into a larger system, whether that‘s a mobile app, web service, or business intelligence dashboard. Using open source tools and open formats in your ML pipeline makes it much easier to plug into different environments." > > > - Luis Ceze, CEO of OctoML The open and collaborative nature of open source is also well aligned with the culture of data science and ML. Practitioners rely heavily on sharing knowledge and building off of each other‘s work. Having a common toolset that everyone is familiar with makes collaboration much more productive. It also makes it easier to leverage the large body of open source models, datasets, and research code to accelerate development. Platforms like Hugging Face hub and Papers With Code have become go-to resources for ML practitioners to discover and share state-of-the-art models. "Machine learning is inherently a collaborative effort," says Jeff Boudier, Product Director at Hugging Face. "No one has a monopoly on the best ideas or the best training data. By making models and datasets open and accessible, we accelerate the pace of research and make it possible to build more powerful applications." From a talent perspective, open source tools are more attractive to many of the best engineers because skills learned on widely-used platforms are highly transferable. A 2019 survey by DigitalOcean found that 71% of developers say open source experience is important to gaining future employment. Companies that rely too heavily on proprietary solutions often struggle to recruit and retain top ML talent. "Working on open source projects is one of the best ways for machine learning engineers to demonstrate their skills and grow their careers," says Vida Williams, former head of AI Infrastructure at Airbnb. "When hiring, I always look for candidates who have made meaningful contributions to popular tools and libraries. It‘s a great signal of their technical ability as well as their commitment to giving back to the community." ## The Open Source MLOps Toolkit So what are the key open source tools that enable MLOps today? Let‘s break it down by capability: For model development, the most widely used open source libraries are: - **TensorFlow** (backed by Google) and **PyTorch** (backed by Facebook) for deep learning - **scikit-learn** for general ML - **XGBoost, LightGBM, CatBoost** for gradient boosting - **spaCy, Hugging Face Transformers, Flair** for NLP These tools provide the core building blocks for creating all types of ML models and are supported by rich ecosystems of extensions and pre-trained models. They also offer flexibility to work at different levels of abstraction, from manual tweaking to automated training and hyperparameter tuning. When it comes to tools for managing ML pipelines and workflows, there are a number of popular open source projects: - **Kubeflow** makes it easier to deploy and manage TensorFlow models on Kubernetes - **MLflow** provides a platform for the complete machine learning lifecycle, including experimentation, reproducibility, deployment, and a central model registry - **DVC** is a version control system for machine learning projects - **Airflow, Luigi, Argo** are workflow orchestration tools for managing complex pipelines - **Feas, Tecton, Feast** are feature stores that help manage and serve model features These tools bring DevOps-style collaboration and automation to ML development. Teams can define end-to-end ML workflows, track experiments, version models and features, and reproduce and audit model lineage. This level of manageability is essential when you have multiple people working on a model and need to maintain it in production. In the deployment phase, containerization technologies like **Docker** and orchestration platforms like **Kubernetes** have become the standard for encapsulating and deploying ML applications at scale. Tools like **Seldon Core, KFServing, BentoML, Clipper, TorchServe** build on top of these foundational technologies to provide higher-level abstractions and workflows specifically designed for deploying trained models in production. They handle concerns like model versioning, A/B testing, scaling, and monitoring out of the box. Once models are deployed, monitoring and observability tools are critical for detecting data drift, ensuring model performance, and troubleshooting issues. Open source tools like **Prometheus** and **Grafana** are widely used for collecting metrics and building dashboards to keep tabs on ML deployments in production. Emerging tools like **WhyLabs, Arize AI, Fiddler, and Evidently** provide even richer model monitoring and analytics capabilities designed for ML use cases. They can track data drift, model accuracy, feature importance, and other key indicators of model health. By combining these open source tools across different phases of the ML lifecycle, organizations can build highly capable and customized MLOps platforms. For example: - **Uber** built its Michelangelo platform on Apache Spark, HDFS, Samza, Cassandra, MLflow, and other open source tools to power ML use cases across the company - **Netflix** uses a combination of Jupyter notebooks, Airflow, gRPC, TensorFlow, PyTorch and Kubernetes to enable various ML applications for personalization, content tagging and more - **Spotify** created the open source Kubeflow Pipelines platform for running its TensorFlow-based ML workflows on Kubernetes - **Iguazio**‘s open source MLRun framework integrates with Kubeflow and automates the ML pipeline from research to production While open source tools provide great power and flexibility, it‘s worth noting that wielding them effectively still requires significant in-house skills and effort. "Assembling an open source MLOps stack is not for the faint of heart," cautions David Aronchick, Head of Open Source ML Strategy at Azure. "You need a strong engineering team that understands both the ML and infrastructure concerns. Pre-integrated platforms will get you started faster. But you‘ll likely still want to leverage open source for customization and portability." For those with the requisite expertise, building on top of modular open source components provides the best balance of control and agility. Let‘s look at a real-world example. When **Stripe** wanted to create an ML platform to support product use cases like fraud detection, it turned to Kubeflow as a starting point. Stripe relies on Google Kubernetes Engine (GKE) as a foundation for much of its software infrastructure. Data scientists at the company were already using TensorFlow. So Kubeflow was a natural fit. Stripe extended Kubeflow with custom components to handle needs like backtesting and bias evaluation for its models. It also integrated Apache Airflow for choreographing model training, evaluation, and deployment steps. This approach allowed Stripe to get up and running quickly with a platform aligned to its existing architecture while still being able to inject its own requirements into the process. "Because most of the components are loosely coupled and communicate via Python, it‘s been very easy to extend the platform to support new use cases," says Rob Story, Product Manager for Stripe‘s ML platform. ## The Limits of Proprietary MLOps Solutions By contrast, commercial MLOps platforms often try to be an all-in-one solution, covering everything from data ingestion to model building to deployment. Companies claim this approach is simpler than stitching together open source tools. But the reality is these proprietary platforms are often quite complex, overly opinionated, and hard to customize. Trying to be good at everything is extremely difficult and most proprietary tools inevitably have gaps in functionality or poor support for use cases that fall outside a narrowly defined "happy path." When ML teams run into limitations, they‘re stuck waiting and hoping the vendor will build the needed capabilities to extend the product. > "I‘ve evaluated a number of commercial ML platforms and they always overpromise and underdeliver. They demo well for simple use cases. But as soon as you try to do anything remotely complex or unique to your business, you quickly hit a wall." > > > - Jeremy Hermann, Sr. Engineering Manager at Codecademy Integrating a proprietary platform with existing systems like your data warehouse or CI/CD pipeline is also much harder than with open source tools that are designed for interoperability. Debugging is more difficult too when you can‘t see what‘s going on under the hood. With open source, you always have the ability to dig into the underlying code to investigate issues. Getting locked into a proprietary MLOps platform can be quite risky. Setting aside the dependence on a vendor‘s product roadmap, commercial platforms also have a habit of unexpectedly changing pricing and licensing in ways that can dramatically increase costs. Vendor pricing strategies like charging based on API calls can lead to astronomical bills as usage grows. "I‘ve seen companies spend upwards of $500K per year on commercial ML platforms that they thought would be much cheaper," says Ernest Chan, data scientist and author. "Meanwhile, open source alternatives like MLflow are free and just as capable for most use cases." Migration challenges are another concern with proprietary tools. Because they are not built around open standards, it can be very difficult to move models and pipelines to a new system if you need to switch vendors. Code and data get stuck inside walled gardens. Whereas with open source, you can often swap one tool for another with minimal changes. The model code itself remains fully portable. ## When to Consider Proprietary Solutions That said, there are some situations where using a proprietary MLOps platform could make sense. For organizations that have little in-house technical expertise, especially when it comes to infrastructure and DevOps, a managed solution may be appealing. The caveat is that these companies will likely still need outside help to be successful with ML in the long run. Canned functionality can only take you so far. Another case is when a team is building simple proof-of-concept models or small-scale solutions that aren‘t mission critical. For quick experiments and one-off analyses, an off-the-shelf GUI tool may be enough to get started without a lot of engineering overhead. But these tools tend to break down as projects get more complex and teams grow. Some organizations also turn to proprietary solutions for their first foray into MLOps with the intention of migrating to an open source stack once they have more experience under their belts. The conventional wisdom is to start small, learn the ropes with a bit of vendor hand-holding, and then take off the training wheels. However, this can be a risky approach if you don‘t plan sufficiently for extensibility and portability from the start. ## Taking Control of Your ML Destiny For organizations that aim to make machine learning a core part of their business, investing in an open source MLOps stack provides significant advantages. Building on open standards makes it much easier to hire and collaborate, customize tools to your needs, and adapt your infrastructure as technology evolves and new use cases emerge. Moreover, being able to fully control and understand your ML platform is invaluable when you need to build secure, compliant, and auditable ML pipelines. You can implement your own best practices rather than being constrained by a vendor‘s predefined workflows and processes. Major cloud providers have recognized the strategic importance of open source and are now making it a key part of their ML offerings. AWS SageMaker, Azure ML, and Google Cloud AI Platform all support popular open source frameworks and tools. Even the most proprietary-leaning vendors are being forced to play nicely with open source to stay relevant. In the end, while proprietary MLOps solutions may help you get a model into production a bit faster, they make it harder to productionize ML at real scale. Companies that embrace open source put themselves in a much better position to innovate and derive long-term value from their ML initiatives. As the famous saying goes, "the future belongs to those who prepare for it today." When it comes to machine learning operations, betting on open source is the surest way to be ready for whatever the future brings. --- Source: [How Open Source Software Powers Modern MLOps](https://33rdsquare.com/how-open-source-beats-proprietary-software-helps-for-mlops/)