ZenML
AzureML Pipelines
All integrations

AzureML Pipelines

Effortlessly Orchestrate Your ZenML Pipelines with the Power of AzureML

Add to ZenML

Effortlessly Orchestrate Your ZenML Pipelines with the Power of AzureML

Seamlessly integrate ZenML with AzureML Pipelines to leverage the robustness and scalability of Microsoft's cloud-based orchestration service. This integration enables you to efficiently build, train, deploy, and manage your machine learning models, streamlining your MLOps workflow.

Features with ZenML

  • Seamless execution of ZenML pipelines on AzureML infrastructure
  • Easy configuration of compute resources for optimal performance
  • Secure access and authentication using Azure credentials or service connectors
  • Integrate with other Azure-based stack components for end-to-end MLOps
  • Support for scheduled pipeline runs using cron expressions or intervals

AzureML Pipelines integration screenshot

Main Features

  • A comprehensive environment for the entire ML lifecycle
  • Scalable computing resources for training and inference
  • Integrated model management and deployment capabilities
  • Collaborative workspace for team productivity

How to use ZenML with AzureML Pipelines


# 1. Install the ZenML `azure` integration
# zenml integration install azure

# 2. Register an AzureML container registry
# zenml orchestrator register <NAME> -f azureml \
#    --subscription_id=<YOUR_AZUREML_SUBSCRIPTION_ID> \
#    --resource_group=<NAME_OF_YOUR_RESOURCE_GROUP> \
#    --workspace=<NAME_OF_YOUR_AZUREML_WORKSPACE> \ 
#    --connector azure_connector

# 3. Update your stack with your new container registry 
# zenml stack update -o <NAME>

from zenml import pipeline, step
from zenml.integrations.azure.flavors import AzureMLOrchestratorSettings

azureml_settings = AzureMLOrchestratorSettings(
    mode="compute-cluster",
    compute_name="my-gpu-cluster",  # Will fetch or create this instance
    size="Standard_NC6s_v3",  # Using a NVIDIA Tesla V100 GPU
    tier="Dedicated",  # Can be set to either "Dedicated" or "LowPriority"
    min_instances=2,
    max_instances=10,
    idle_time_before_scaledown_down=60,
)

@step
def hello_world() -> str:
    return "Hello World!"


@pipeline
def my_pipeline():
    _ = hello_world()


if __name__ == "__main__":
    my_pipeline()
    
   

Additional Resources

Connect Your ML Pipelines to a World of Tools

Expand your ML pipelines with more than 50 ZenML Integrations

  • Amazon S3
  • Apache Airflow
  • Argilla
  • AutoGen
  • AWS
  • AWS Strands
  • Azure Blob Storage
  • Azure Container Registry
  • BentoML
  • Comet
  • CrewAI