Model Deployment and Monitoring
How do you deploy a model in Iguazio?
Iguazio uses MLrun for deploying models as serverless functions (Nuclio) running on Kubernetes. This creates an endpoint that can be accessed from remote. It also has out of the box scaling capabilities (scale up and down) that are based on actual workloads.
Do you support canary deployment?
Yes, Iguazio has an API gateway that is used for canary deployment. You can control the amount of traffic that goes to two different models and gradually promote your new model using the canary deployment approach.
Can I bring my own models to Iguazio?
Yes. Any Python compatible model can be brought into, used, and deployed with Iguazio.
Does Iguazio have model monitoring or drift detection?
Yes. We have model monitoring out of the box. We also provide the ability to kick off alerts when model drift is detected (such as sending a Slack message or starting a re-training pipeline).
Does Iguazio offer any dashboards for model monitoring?
Yes. We have dashboards out-of-the-box for latency, number of predictions, cluster resources, feature distributions, and feature drift.
Do I need a separate environment for Dev, Qa, and Prod?
Yes it is preferable to have separate environments for development, QA, and production. It is feasible to deploy all of these on the same cluster, but this is not recommended.
If I have a Dev, QA, and Prod environment, and I trained my model on Dev, how can I deploy my model to Prod?
Iguazio's development concept is to work with projects. A Project is a container for all your work on a particular activity/application. All the associated code, functions, jobs/workflows, and artifacts are organized within the projects. Projects can be mapped to GIT repositories or IDE project (in PyCharm, VSCode, etc.) which enable versioning, collaboration, and CI/CD. Then once the code has been changed and committed to git you can load it onto the production cluster as part of a CI/CD process. For more information see MLRun projects.
How are models/function versioned? And how can I roll back to a specific version of a mlrun function used in the past?
All models and functions are versioned using a tag system (much like Docker). You can refer to a specific version of a model/function using the specified tag.
If I a have a complex online pipeline, how can Iguazio help me?
Iguazio provides an option to deploy a complex graph using MLRun serving graphs. It allows the building of real-time pipelines that include data processing, advanced model serving, custom logic, and fast access to a variety of data systems. They can then be deployed quickly to production with minimal effort. For more information, see Serving Graph.