Efficiently Store and Share ZenML Artifacts with Azure Blob Storage
Enhance your ZenML workflows by leveraging Azure Blob Storage as a scalable and reliable artifact store. This integration enables seamless storage and sharing of pipeline artifacts, making it ideal for collaborative ML projects and production-grade MLOps.
Features with ZenML
- Seamlessly store and retrieve pipeline artifacts in Azure Blob Storage
- Enable collaboration by sharing artifacts across teams and stakeholders
- Scale storage effortlessly to handle the growing demands of ML projects
- Integrate with other Azure-based stack components for end-to-end MLOps
- Secure access to artifacts using Azure authentication methods
Main Features
- Scalable and durable object storage for unstructured data
- High availability and geo-redundancy options
- Flexible access control and security features
- Cost-effective storage for large-scale ML artifacts
- Seamless integration with other Azure services
How to use ZenML with
Azure Blob Storage
# 1. Install the ZenML `azure` integration
# zenml integration install azure
# 2. Register an Azure artifact store
# zenml artifact-store register <NAME> --flavor azure --path=<PATH_TO_STORAGE>
# 3. Register a stack with the new artifact store
# zenml register stack <STACK_NAME> -a <NAME> -o default --set
from typing import Annotated
from zenml import pipeline, step
from zenml.client import Client
@step
def hello_world() -> Annotated[str, "my_first_artifact"]:
return "Hello World!"
@pipeline
def my_pipeline():
_ = hello_world()
if __name__ == "__main__":
my_pipeline()
# Fetch the artifact and print it
print("Result: ", Client().get_artifact_version("my_first_artifact").load())
This code example demonstrates how to set up and use the Azure Blob Storage integration with ZenML:
- Install the Azure integration
- Register an Azure Artifact Store by using the path to your blob storage
- Register a ZenML stack with the Azure Artifact Store
- Use the
@step and @pipeline decorators to define a pipeline to store artifacts in the Azure Artifact Store.
Additional Resources
Read the guide for setting up a full Azure stack
Read the ZenML Azure Blob Storage Artifact Store Documentation
Azure Blob Storage documentation
.png)
