![Why ML in production is (still) broken - [#MLOps2020]](https://cdn.prod.website-files.com/65264f6bf54e751c3a776db1/65316d2bc051413294285f1e_mlopsworldthumbnail.png)
.jpeg)
.jpeg)
Imagine a world where machine learning (ML) models seamlessly transition from concept to production, effortlessly scaling to meet real-world demands. Welcome to the realm of Machine Learning Operations or MLOps – the game-changer in the AI landscape. As businesses increasingly harness the power of ML, the need for streamlined workflows has never been more critical. Enter Google Cloud Vertex AI, a powerhouse platform that's revolutionizing how we approach MLOps. From turbocharging data preparation to simplifying model deployment, Vertex AI is the secret weapon for organizations looking to unlock the full potential of their ML initiatives. In this blog, we'll dive into the MLOps revolution and explore how Vertex AI's arsenal of tools – including pre-trained models, automated pipelines, and real-time monitoring – can transform your ML journey. Whether you're an MLOps novice or a seasoned pro, discover how Google Cloud's robust ecosystem, featuring Cloud Connected services and Cloud Armor security, can elevate your ML game to new heights. Get ready to reimagine your approach to machine learning – the future of MLOps awaits!
What is MLOps?
Like DevOps and its benefits for software development, MLOps is a concept developed to enhance the development of machine learning systems. MLOps encompasses every stage of the machine learning lifecycle, from building and deploying to serving and monitoring ML models. Using the right platform, processes, and people helps businesses bring models into production faster and with higher success rates. Utilizing a unified platform such as Google Cloud Vertex AI, which encompasses Google Cloud AI services and Google Vertex AI, can greatly enhance this process. By making Google Cloud a central service, you can ensure the strength and seamless integration of your cloud services, particularly when working with Cloud Storage Object files.
Together, these features can reduce the loss in model performance and manage the growing complexity of operating machine learning systems when used correctly. With the help of multimodal models and an extensive application performance suite, this lowers overhead and operational expenses for the company while also opening up new income streams and the use of sophisticated analytics and ML-driven decision-making.
GCP Basics: Raw Services for MLOps
In machine learning operations (MLOps), deploying and maintaining ML models requires the right infrastructure and tools. The Google Cloud Platform (GCP) offers services to support the entire ML lifecycle. This blog will focus on core GCP services essential for successful MLOps, including Google Artifact Registry, Google Cloud Storage, Google Cloud Build, Compute Engine, and Google Kubernetes. Understanding how to deploy application platforms and manage application logs is crucial for maintaining application health and identifying errors.
Google Artifact Registry
In MLOps, it's crucial for you to version and manage artifacts like trained models, data pipelines, and containers. Google Artifact Registry provides you with a secure solution for storing your ML models and related artifacts. By integrating Artifact Registry with your CI/CD pipelines, you can automate the deployment of ML models to production environments. Additionally, leveraging assistants for application development can further streamline your workflow.
Google Cloud Storage
Your MLOps pipeline depends on Google Cloud Storage, and data is essential for your machine learning activities. It acts as a centralized location for storing model binaries, training data, and other crucial resources. Its seamless connection with other GCP services, such as AI Platform and BigQuery, allows you to handle and process your data effectively. Furthermore, versioning is supported by Cloud Storage, guaranteeing the preservation of every version of your dataset—a crucial component for model repeatability and compliance. Your Model Garden implementations and other cloud computing needs, such as Cloud Storage Object storage, will benefit from this service.
Google Cloud Build
In order to guarantee continuous and dependable ML model deployment in your MLOps pipeline, automation is crucial. You may link Google Cloud Build with other GCP services to build automated CI/CD pipelines for your machine-learning operations. This enables the creation of Docker images, testing, and model deployment to cloud run or Google Kubernetes Engine production settings. You may cut down on mistakes and accelerate the time to market for your machine learning products by automating these stages. Furthermore, you may use application platform tools, database services, and workflow orchestration services to efficiently handle large-scale deployments.
Compute Engine
In order to perform large ML workloads in MLOps, compute engines are essential. It offers the adaptability and strength required for conducting inference, training models, and preparing data. To build a strong MLOps pipeline, you can also combine Compute Engine with other GCP services like Cloud Storage and Cloud Build. Compute Engine improves training performance when used in conjunction with pre-trained models, guaranteeing smooth interaction with AI-powered applications and console tool features.
Google Kubernetes Engine (GKE)
For MLOps on GCP, GKE is a crucial service, particularly for installing and scaling ML models in production. You may take advantage of Kubernetes' automatic scaling, rolling updates, and self-healing features by containerizing your machine learning models and deploying them on GKE. In order to provide a streamlined process for deploying and managing ML models in a scalable and dependable way, GKE also connects with other GCP services, such as Cloud Build for CI/CD and Artifact Registry for managing model containers. The deployment process is further strengthened by the freedom to employ bespoke containers, which also simplifies the management of prebuilt containers.
MLOps on GCP: Vertex AI Suite of Tools
For nearly a decade, Google Cloud has been at the forefront of developing cutting-edge MLOps tools and solutions, and now, with Google Cloud Vertex AI, you're equipped with a powerful platform to build, deploy, and scale ML models faster than ever. Vertex AI abstracts the complexities of raw services into a unified platform, allowing you and your Data Science team to focus on what really matters—your work—without getting bogged down by the underlying infrastructure.
However, this convenience does come with a price, so it's important to evaluate your organization's needs carefully. Whether you're focused on Infrastructure-as-Code, CI/CD pipelines, continuous training pipelines, or prediction services—whether batch or online—Vertex AI has something to offer. You'll also find powerful tools for task automation with schedulers and analytics to drive insights. Your specific approach will depend on your organization's goals and the demands of your ML use case, but with Vertex AI, you're well-equipped to tackle it all.
Vertex AI Pipelines
Implementing a robust MLOps solution can be challenging, but tools like Google Cloud Vertex AI offer you a streamlined approach. With Vertex AI Pipelines, you can simplify the process, making it easier for you and your team of data scientists and ML engineers to accelerate your machine learning initiatives using a cohesive platform. The availability of pre-trained models and multimodal model integration further enhances your capabilities, while the GCP application performance suite ensures that all components work together seamlessly.
Vertex AI Custom Jobs
In addition to its powerful automation tools, Google Cloud Vertex AI also supports custom jobs, giving you the ability to run specialized tasks that go beyond the capabilities of pre-built models and AutoML. With custom jobs, you have the flexibility to train models using your own code and environments, making them perfect for complex ML tasks that require high customization. This flexibility is further enhanced by automated tools and database services, streamlining your entire MLOps process.
Vertex AI’s Role in Specialized ML Tasks
- Custom Training Environments: With Vertex AI Custom Jobs, you can define your own training environments, including specific frameworks, libraries, and dependencies. This is particularly valuable for specialized ML tasks that need custom configurations.
- Scalable Resources: You can execute custom jobs across multiple GPUs or TPUs, speeding up the training of large models.
- Job Orchestration: Integrate these custom jobs into Vertex AI Pipelines for smooth orchestration within your broader ML workflows, ensuring everything runs together seamlessly.
- Flexibility: Vertex AI provides the flexibility to customize your ML models, whether you're working with complex neural networks, unique datasets, or cutting-edge research. By utilizing app platforms and cloud-native wide-column databases, you can push the boundaries of what these custom tasks can achieve.
Challenges in Adopting MLOps with GCP
When it comes to scaling machine learning processes, many companies like yours turn to GCP for their MLOps solutions due to its robust infrastructure and extensive toolkit. However, using GCP isn't without its challenges. In this blog, we'll walk you through potential roadblocks such as gaining departmental buy-in, establishing robust pipelines, integrating with external services, fostering cross-team collaboration, and managing expenses effectively.
We'll also discuss the importance of cloud security and how Cloud Armor Security can protect your business in a multi-cloud environment, helping you secure operations and ensure long-term success.
Adoption Within the Data Science Department
Challenge:
Due to unfamiliar tools and processes, data scientists may need to learn about adopting MLOps.
Solutions:
- Training: Invest in upskilling on GCP tools and MLOps best practices, focusing on Generative AI Studio and analysis tools.
- Gradual Adoption: Implement MLOps incrementally, starting with quickstarts and labs to familiarize teams with new workflows.
- Cross-Functional Teams: Encourage collaboration between data scientists, ML engineers, and DevOps.
Hard to Write Pipelines
Challenge:
Developing and managing ML pipelines on GCP can be complex, especially for teams new to cloud-based MLOps.
Solutions:
- Pre-Built Templates: Use GCP’s templates to simplify pipeline development.
- GCP Experts: Collaborate with GCP professionals for pipeline design, mainly focusing on video analysis and real-time analytics.
- Tooling: Leverage GCP’s monitoring and debugging tools, including application logs management and application error identification.
Hard to Integrate External Services
Challenge:
Integrating external services with GCP can be complex and costly.
Solutions:
- GCP Marketplace: Use pre-configured integrations.
- API Gateways: Secure and manage external interactions, focusing on predictive analytics and Secure video meetings for enhanced collaboration.
- Optimize Data Transfer: Reduce egress costs by optimizing data transfers, mainly when dealing with cloud-native relational databases and document database services.
Cross-Project, Cross-Team, Cross-Region Collaboration
Challenge:
Collaboration across projects, teams, and regions can be complex due to differences in configurations and availability.
Solutions:
- Centralized Management: Use GCP’s resource management tools for consistency, ensuring seamless collaboration through collaboration for teams and modern collaboration practices.
- Standardized Practices: Implement uniform MLOps guidelines, including workflow orchestration service for consistency.
- Regional Optimization: Minimize latency and ensure data compliance, mainly when dealing with warehouses for business agility and agnostic edge solution deployments.
Expensive
Challenge:
MLOps on GCP can be costly, especially for large-scale workloads.
Solutions:
- Cost Monitoring: Use GCP’s budgeting tools to control expenses.
- Workload Optimization: Choose efficient machine types and minimize unnecessary pipeline runs, mainly when using minimal downtime migrations and the cycle of APIs for optimization.
- Data Management: Implement policies to manage storage costs, especially when dealing with Cloud-native document databases and Cloud-native wide-column database storage options.
While GCP offers solid tools for MLOps, you'll need to address certain challenges to unlock their full potential. By adopting the right strategies, you can build scalable and efficient MLOps processes that drive innovation and boost operational efficiency. Leveraging features like automatic cloud resource optimization and Cloud Armor Security will help protect your operations.
Choosing the Right Services on GCP
When deciding which services to use for coordinating your ML workloads on the Google Cloud Platform (GCP), understanding the importance of orchestration is key. Orchestration ensures that your machine learning workflows are managed and executed efficiently, making it easier to streamline complex processes and automate repetitive tasks. As your workflows grow more intricate and your need for customization and automation increases, effective orchestration becomes even more crucial.
It allows you to manage dependencies, dynamically scale resources, and seamlessly integrate different parts of your ML pipeline. By comparing Compute Engine, Google Kubernetes Engine (GKE), and Vertex AI Pipelines, you'll gain insights into how each service supports orchestration and how to choose the one that best fits your needs. Whether you're focused on app migration, development, or performance, understanding these options will help you make the right decision for your projects.
Deciding Between Compute Engine, GKE, and Vertex AI Pipelines
When choosing a Google Cloud Platform (GCP) service to orchestrate your ML workloads, consider key factors like the availability of pre-trained models, the power of AI-driven apps, and the flexibility to use custom containers. These elements will guide you in selecting the right service to meet your specific needs efficiently.
Step-by-step Tutorial: Using an open source framework to optimize your MLOps on GCP
Optimizing your Machine Learning Operations (MLOps) is crucial for managing and scaling complex processes in today's fast-paced ML environment. While Google Cloud Platform (GCP) provides a strong foundation, handling these processes can be challenging. By incorporating an open-source framework like ZenML, you can simplify and streamline your MLOps on GCP, allowing you to focus on what truly matters—developing, deploying, and managing high-performance ML models.
In this section, you'll discover how to set up and refine your MLOps pipeline on GCP using ZenML, making your machine learning deployments more efficient and scalable. We'll also dive into leveraging Generative AI workflows, Model Registry services, and the integration of sample prompts to further enhance your model development and deployment process. Whether you're aiming to boost efficiency or scale your operations, ZenML has the tools to help you succeed.
What is ZenML?
ZenML is an open-source MLOps framework that creates portable, production-ready machine learning pipelines. It offers extensive integration options, decoupling infrastructure from code to enable seamless development and deployment. This simplification allows machine learning engineers to focus on strategic growth rather than infrastructure headaches. Coupled with Kubeflow, ZenML enhances project adaptability and scalability, providing a streamlined pathway for rapid ML model deployment to production, utilizing Google Cloud's AI and Google AI Studio capabilities.
How ZenML Integrates with GCP Services
The diagram below illustrates how ZenML, an open-source MLOps framework, seamlessly integrates with key Google Cloud Platform (GCP) services to streamline your machine learning workflows. By connecting with GCP components like Vertex AI Pipelines, Kubernetes Engine, and Cloud Storage, ZenML automates the orchestration, deployment, and management of your ML models, allowing you to scale and manage your ML operations with greater efficiency. Additionally, automated tools, a code development platform, and analytics functionalities further enhance this powerful integration, making it easier for you to focus on innovation.
Starting with a basic Google Cloud stack
The easiest cloud orchestrator is Google Cloud Vertex AI, which runs on a public cloud. ZenML also offers the flexibility to work with Kubernetes or raw virtual machines (VMs), but for this example, we'll focus on using Vertex AI Studio. This approach benefits from cloud-native automation and cloud migration strategies.
When using Vertex AI, we require a method to package and transfer your code to the cloud so that ZenML can function correctly. ZenML uses Docker to accomplish this. Whenever you initiate a pipeline with a remote orchestrator, ZenML creates an image for the entire pipeline (and optionally for each pipeline step based on your configuration). This image contains all the code, requirements, and other necessary components to execute the pipeline steps in any environment. ZenML then uploads this image to the container registry specified in your setup, and the orchestrator pulls the image when it's ready to run a step.
To summarize, here is the broad sequence of events that happen when you run a pipeline with such a cloud stack:
- Firstly, the user runs a pipeline on the client's machine. This executes the run.py script where ZenML reads the @pipeline function and understands what steps must be executed.
- The client asks the server for the stack info, which returns it with the configuration of the cloud stack.
- Based on the stack info and pipeline specification, the client builds and pushes an image to the container registry. The image contains the environment for executing the pipeline and the steps' code.
- The client creates a run in the orchestrator. For example, the Vertex AI orchestrator creates a virtual machine in the cloud with commands to pull and run a Docker image from the specified container registry.
- The orchestrator pulls the appropriate image from the container registry as it executes the pipeline (each step has an image).
- As each pipeline runs, it stores artifacts physically in the artifact store. Of course, this artifact store must be some form of Cloud Storage Object storage.
- As each pipeline runs, it reports status to the ZenML server and optionally queries the server for metadata.
Connect your GCP to ZenML in one-clickZenML simplifies the process of incorporating GCP services into your ML workflows. Here's astep-by-step guide to setting up GCP within ZenML:Install Python
- Download and install the correct Python version if you don’t have it.
- Ensure it's between versions 3.8 and 3.11 for compatibility.
Ensure you have Python installed
Set up a Virtual Environment
- Creating a virtual environment for your project is good practice to avoid conflicts with other Python software packages.
Creating a Virtual Environment:
To learn more about how to activate Python virtual environments.
Connect to ZenML
Login to your ZenML Pro tenant
- Do the required authorization.
The example below will walk you through training a model and creating a pipeline using the breast cancer dataset. It's meant to be simple and quick, providing a practical introduction to the process. In real-world situations, you will probably work with larger datasets and more complex models, but this is a good place to start.
Register a remote stack:
A stack configures how a pipeline is executed. Learn more. Connect your Cloud to deploy your ZenML pipelines in a remote stack.
In ZenML, the stack is a fundamental concept that represents the configuration of your infrastructure. In a typical workflow, creating a stack requires you first to deploy the necessary pieces of infrastructure and then define them as stack components in ZenML with proper authentication.
Especially in a remote setting, this process can be challenging and time-consuming, and it may create multi-faceted problems. This is why we implemented the stack wizard feature, which allows you to browse your existing infrastructure and use it to register a ZenML cloud stack.
How to use the Stack Wizard?
Check out our quick 2-minute tutorial video and detailed documentation on the Stack Wizard. Discover how easy it is to get started!
Run the pipeline in the new stack
Install the integrations
- Install the required integrations to run pipelines in your stack
Run the training pipeline:
Go to your ZenML Pro account and view the pipeline status:
Detailed Breakdown of a Machine Learning Pipeline:
- data_loader: Loads the initial dataset (
DataFrame). This step involves fetching the raw data that will be used for training and testing the model. - data_splitter: Splits the dataset into training and test subsets (
my_data_subset,raw_dataset_tst,raw_dataset_trn). This step ensures the data is divided appropriately to accurately evaluate the model's performance. - data_preprocessor: Processes raw datasets into training and test sets (
dataset_trn,dataset_tst) and defines the preprocessing pipeline. This step includes cleaning, transforming, and normalizing the data to make it suitable for model training. - model_trainer: Trains a
RandomForestClassifiermodel using the processed training dataset, focusing on model accuracy and robustness. This step involves using the prepared data to train the machine learning model and optimizing it for the best performance. - model_evaluator: Evaluates the trained model using the test dataset, providing an evaluation metric (float). This step assesses the model's performance on unseen data to ensure it generalizes well and performs accurately.
- model_promoter: Decides on model promotion based on the evaluation result (boolean).
This pipeline loads, splits, preprocesses data, trains, evaluates, and potentially promotes a machine learning model. It focuses on efficient model training, ensures model robustness, and includes sentiment analysis to enhance the model's performance and applicability.
ZenML provides a robust framework for optimizing MLOps on GCP. It offers seamless integration with GCP services, structured project management, and easy configuration. Following this tutorial, you can quickly set up and manage ML pipelines on GCP, leveraging ZenML’s capabilities to streamline your machine learning operations. Whether you’re a data scientist or ML engineer, ZenML makes it easier to focus on building and deploying models without getting bogged down in infrastructure complexities.
Case Study: ADEO Leroy Merlin + Bravo - Accelerating MLOps with ZenML on GCP
“ZenML has proven to be a critical asset in our machine learning toolbox, and we are excited to continue leveraging its capabilities to drive ADEO's machine learning initiatives to new heights.”
— François Serra, ML Engineer / ML Ops / ML Solution Architect at ADEO Services
ADEO, a prominent name in the global retail sector, has embarked on a data-driven transformation to enhance its competitive edge. Recognizing the need for a scalable and efficient machine learning (ML) pipeline, ADEO leveraged ZenML to transition from manual, fragmented processes to an agile, automated MLOps setup on the Google Cloud Platform (GCP). This shift significantly reduced their time-to-market and improved deployment efficiency, particularly by integrating prebuilt containers and Automated tools.
In a separate case study, Brevo partnered with ADEO to further streamline their MLOps framework. Brevo's expertise was crucial in optimizing the ML pipeline, contributing to a more efficient and scalable solution that aligns with ADEO's broader digital transformation goals.
ZenML and GCP Integration
ADEO encountered the challenge of standardizing its machine learning operations across different data science teams, each with its preferred tools and platforms. Some teams favored Google Kubernetes Engine (GKE) for containerized workloads, while others relied on raw virtual machines (VMs), and some were drawn to the managed services of Google Cloud Vertex AI. This variety made it difficult to streamline ML workflows across the board.
Enter ZenML—the flexible, framework-agnostic solution ADEO needed. By implementing ZenML on the Google Cloud Platform (GCP), ADEO could unify its ML pipelines across all three environments—GKE, raw VMs, and Vertex AI Studio. This allowed each team to continue working in their preferred environments while ensuring consistency and reproducibility across the organization. With ZenML, ADEO could automate the construction, versioning, and deployment of ML models, keeping the ML lifecycle seamless and efficient, no matter what infrastructure their teams chose.
Structuring Projects in ZenML
In ADEO's journey to streamline and standardize their machine learning operations using ZenML on Google Cloud Platform (GCP), ZenML Pro has been a game-changer. As your teams transitioned from fragmented processes to a unified MLOps setup across GKE, raw VMs, and Vertex AI, ZenML Pro provided the infrastructure needed to centralize and manage operations across multiple teams.
Whether focusing on Fraud Detection, Recommendation Systems, or Large Language Models (LLM), each team operates independently across various GCP projects and regions while maintaining a consistent and scalable MLOps framework. ZenML Pro empowers you to efficiently manage and orchestrate ML workflows, using distinct "stacks" tailored to different GCP projects and regions. This flexibility allows teams to choose the best infrastructure for their needs while you maintain centralized control over the entire MLOps lifecycle, ensuring consistency, reproducibility, and scalability.
Conclusion
Using ZenML and other tools to implement MLOps on the Google Cloud Platform may completely change the way you handle machine learning operations. Through the use of GCP's powerful infrastructure and technologies, like Google Cloud Vertex AI, Google Kubernetes Engine, and ZenML, you can improve collaboration, automate pipelines, and simplify intricate machine learning procedures. This connection gives you a competitive edge by streamlining your operations and speeding up your time to market.
Adopting a thorough MLOps framework on GCP will put your organization in a position to develop more quickly and effectively. Additionally, you can improve your operations even further while cutting expenses and budgeting more efficiently by using code samples and pre-trained models from Google Cloud AI services.
Now, it's your turn to take action. Start integrating these powerful tools into your projects to accelerate development and stay ahead of the competition. Take the next step in your MLOps journey by deploying your first stack today. Follow our GCP quickstart guide to get started: Deploy a Cloud Stack.
❓FAQ
1. How do you build an ML pipeline in GCP?
To set up a machine learning (ML) pipeline in Google Cloud Platform (GCP), store your datasets in Google Cloud Storage, preprocess the data using Dataflow or Dataprep, train your model using Vertex AI Studio, and finally deploy it to make predictions. Automate the process using Cloud Composer (Airflow) or Kubeflow Pipelines for a scalable and consistent workflow. Check out the ZenML docs for an easy GCP implementation.
2. Which GCP service can perform machine learning tasks such as training and prediction in the cloud?
Vertex AI Studio is the GCP service that performs machine learning tasks such as training and prediction in the cloud. It offers a comprehensive set of tools and APIs for building, deploying, and managing machine learning models. It supports custom model training, hyperparameter tuning, and prediction deployment, making it an ideal choice for end-to-end machine learning workflows on Google Cloud. Additionally, Generative AI Studio provides advanced features for AI-driven solutions and 5G. AI-driven solutions that integrate seamlessly into your workflow.
3. Which cloud platform is best for MLOps?
The best cloud platform for MLOps depends on your specific needs. Google Cloud Platform (GCP) is often considered a top choice due to its comprehensive set of tools for machine learning and MLOps. GCP's Vertex AI offers integrated model training, deployment, and management capabilities and supports pipelines and CI/CD for ML workflows. GCP's Kubernetes Engine (GKE) and Kubeflow also provide robust solutions for scalable MLOps practices. Strong contenders include Amazon Web Services (AWS) with its SageMaker suite and Microsoft Azure with Azure Machine Learning, offering extensive MLOps capabilities. Integrating agnostic edge solutions and analytics solutions further enhances the platform's capabilities.
4. What is MLOps Google Cloud?
MLOps on Google Cloud refers to managing the machine learning lifecycle efficiently using DevOps principles and automation. Vertex AI Studio in GCP supports MLOps with tools for building, deploying, and managing machine learning models at scale. It ensures robust, scalable, and maintainable models and provides additional support for 360-degree patient view, 3D visualization, and advanced reasoning capabilities.
