ZenML 0.70.0: Major Updates & Important Migration Notes

We're excited to announce the release of ZenML 0.70.0, which brings significant improvements to artifact versioning, scalability, and metadata management. However, due to substantial database schema changes, this upgrade requires careful attention and preparation.

⚠️ Important Upgrade Notice

This release includes major database schema changes and migrations. Before proceeding with the upgrade:

1. Create a complete backup of your production database
2. Test the upgrade process in a non-production environment first
3. Plan for potential downtime during the migration

🚀 Key Improvements

Enhanced Artifact Versioning

We've substantially improved how ZenML handles artifact versions:

• New batch processing capabilities for artifact version requests, resulting in faster execution times
• Support for multiple versions of the same artifact (e.g., model checkpoints)
• Improved user experience in both the ZenML UI and API interactions

Scalability Boost

Performance and scalability have been major focus areas in this release:

• Reduced server request frequency
• Server-side artifact version incrementing
• Optimized processing for improved speed at scale

Unified Metadata Management

We've simplified metadata handling with a new unified approach:

• Introduced the consolidated log_metadata method
• Support for logging run metadata across:
  • Artifact versions
  • Model versions
  • Steps
  • Runs

New Filtering Capabilities

The new oneof operator brings more flexible filtering options:

‍# Filter by IDs (UUID type)
PipelineRunFilter(id='oneof:["uuid1", "uuid2"]')

# Filter by tags or string attributes
PipelineRunFilter(tag='oneof:["cats", "dogs"]')

Documentation Enhancements

Our documentation has received a significant update:

• Restructured for better navigation
• New sections covering finetuning
• Additional LLM/ML engineering resources
• Improved getting started guides

Bug Fixes

This release addresses several issues:

• Resolved in-process main module source loading problems
• Various other stability improvements

📋 Upgrade Checklist

Before upgrading to version 0.70.0:

• Create a full backup of your ZenML database
• Test the upgrade in a staging environment
• Review your current workflows for any potential compatibility issues
• Plan for maintenance window if upgrading production systems
• Ensure all team members are aware of the upcoming changes

Looking Forward

These improvements lay the groundwork for even better ML pipeline management and scalability. We're committed to making ZenML more robust and user-friendly with each release.

Stay tuned for more updates, and don't hesitate to reach out to our community channels if you need any assistance with the upgrade process.

Happy MLOps!