Integrate Google Cloud Vertex AI Pipelines with ZenML

Streamline your MLOps workflows on GCP with ZenML and Vertex AI Pipelines

Enhance your machine learning operations by leveraging the power of Vertex AI Pipelines orchestration through ZenML. This integration enables you to run production-ready, scalable ML pipelines on Google Cloud Platform, taking advantage of the fully managed serverless infrastructure and intuitive UI for tracking pipeline runs.

Features with ZenML

Seamlessly integrate ZenML pipelines with Vertex AI Pipelines for end-to-end ML workflows on GCP
Easily deploy and scale your pipelines using Vertex AI's managed serverless infrastructure
Track and monitor pipeline runs through the intuitive Vertex AI UI, accessible directly from ZenML
Leverage GPU acceleration for compute-intensive steps in your ZenML pipelines
Schedule recurring pipeline runs using Vertex AI's native scheduling capabilities

‍

Google Cloud Vertex AI Pipelines integration screenshot

Main Features

Fully managed serverless infrastructure for running ML pipelines at scale
Intuitive UI for visualizing and monitoring pipeline runs and logs
Native support for GPU-accelerated workloads
Flexible scheduling options for recurring pipeline runs
Seamless integration with other GCP services and tools in the Vertex AI platform

‍

How to use ZenML with Google Cloud Vertex AI Pipelines


zenml integration install gcp
zenml stack set ...


from zenml.integrations.gcp.flavors.vertex_orchestrator_flavor import VertexOrchestratorSettings

# Choose an accelerator to run on
vertex_settings = VertexOrchestratorSettings(
                node_selector_constraint=(
                    "cloud.google.com/gke-accelerator",
                    "NVIDIA_TESLA_P4",
                )
            )

@pipeline(
    settings={
        "orchestrator.vertex": vertex_settings,
    }
)
def vertex_pipeline():
    ingest_data()
    train_model()
    evaluate_model()

# Run the pipeline
vertex_pipeline()

Additional Resources

Using Vertex with a full google stack Learn more about Vertex AI Pipelines in the GCP documentation