Hello, hello
I've been thinking a lot about the positioning of ZenML in the wider MLOps + LLMOps ecosystem. In a lot of ways, we're still seen as a pipeline/orchestration tool (similar to Airflow). While I think pipelines are a big part of ZenML, the more we've built and talked to users, the more I realize it's more the abstraction of infrastructure rather than the pipeline that makes ZenML different. We're doubling down on that by making it even easier to connect existing infrastructure into the tool.
Do you have any thoughts on the value proposition of ZenML? If yes, please reply to this email and let me know. Until then, here are the biggest updates last month:
🔧 Fine-tune Llama 3.1 on your MLOps stack
Llama 3.1 was released last week, a landmark moment in the history of GenAI, when finally an OSS model beat out their proprietary counterparts in most benchmarks. A day later, Alex wrote a step-by-step tutorial on how to finetune it on your own data, with ZenML. The pipeline is very straightforward and to the point, and I'd encourage everyone to try it out. There's instructions on how to set it up with GCP, but this should work on all backends as usual. Try it out!
🎢 Terraform + ZenML: Bridging MLOps and IaC
We've released Terraform modules for all major cloud providers, that let you spin up a simple MLOps stack and register back to your ZenML server. This provides a bridge between the DevOps heavy Infrastructure-as-Code (IaC) world and MLOps. I wrote a blog talking about the benefits of having Terraform-native integrations into ZenML.
👀 Viewing Huggingface Datasets on your ZenML dashboard
If you return a Huggingface dataset using ZenML, you now get an embedded visualization of it in the ZenML dashboard. Stoked to see this get much recognition by being featured on the Huggingface docs, and being shouted out by Julien Chaumond, the CTO of Huggingface himself!
💻 Exciting VS Code extension updates
Have you tried using ZenML Studio, the VS code extension to ZenML? If not, you're missing out! Eric and Chris from Launch School have been shipping updates at lightning speed to the OSS repo, and they've gotten it to a point where it actual has more features for stack component registration than the dashboard! Install the VS Code extension now and check it out - it's really seamless!
😎 The rest...
- AWS, GCP, and Azure are supported now for the 1-click deployment and the stack registration wizard. Registering stacks has never been easier.
- We've gotten rid of pipeline versions (keep an eye on why in the next release).
- ZenML now supports a new databricks orchestrator and model deployer!
- An updated end-to-end computer vision project is updated and live!
- I got interviewed by Infoworld regarding the latest AI report from the White House.
- An AzureML pipelines orchestrator is in the works!
Cheers,
Hamza
Co-founder, ZenML
