Best Practices for Succeeding with MLOps 

Data science is an important skill, but the hard truth is many organizations aren’t seeing the ROI showing that data science work is making a business impact. Yet today, many organizations are still struggling to adopt a holistic approach centered around creating business value. Instead, they are focused on theoretical work.

Here at Iguazio, we recently held a webinar with Noah Gift, founder of Pragmatic A.I. Labs, professor, author and MLOps consultant. In his talk, he provided best practices for succeeding with MLOps and connecting data science to a clear ROI. Below, you can find the main best practices and future trends he presented. The entire webinar is available on-demand, and has more in-depth explanations and examples.

1. Pick the Right Technology Partners and Solutions for You

The right technology partners will help you scale, adapt to future technological changes and be innovative. We recommend having two to three technology partners/solutions in place:

• The first, your foundational solution, which is meant to be your primary product. These are solutions like AWS, Azure or Google Cloud.
• The second solution is a platform that solves a specific problem very well. For example, Iguazio, Splunk, Snowflake or Databricks.
• The third is a long-term R&D investment. For example, frameworks for ML/DL, Kubernetes or edge computing.

How should you choose these primary, secondary and investment partners? While there is no silver bullet for choosing the right platform or technology provider, there is a set of considerations Gift recommends taking into account.

Considerations for choosing your primary technology partner:

• Low costs: to work efficiently and meet the budget
• Agility: to enable flexibility and autonomy
• Popularity: to ensure the tech has a future
• Easy to hire for: to scale your team with relevant skillsets
• Has training and documentation: to build your team’s capabilities over time
• Breadth: to ensure support of multiple features
• Easy to build abstractions on top:  to provide you with innovational freedom

Considerations for choosing your secondary technology partner:

• Solves one problem very well: To help you solve the specific challenge you are encountering and that is preventing you from scaling and meeting your business goals or from performing other tasks you need to do. 
• Easy to hire for
• Provides training and certifications: to scale and train your team
• Integrates with primary platform: to enable your technological stack
• Built on top of a popular technology: to ensure stability, robustness and credibility of the solution

Considerations for choosing your investment technology:

• R&D focus: To support technological innovations like:
  • Deep learning tech
  • Kubernetes
  • Edge computing
  • Pre-trained models (Hugging Face, etc.)
• Easy to hire for
• Provides training

By taking these considerations into account and choosing the right partners, you will be able to build a scalable and robust product that will help you adapt to the future.

2. Leverage Learning Platforms and Certifications

Gift recommends a subscription to a learning platform, to encourage growth within organizations. Through online courses and training, your employees can learn new technologies, improve their knowledge of existing ones and learn information they can share with each other. This not only helps them do their job better, it also helps them become tech evangelists, give monthly tech talks and show internal demos of solutions.

Gift recommends subscribing to at least two platforms. Such platforms include O’Reilly, Coursera and Udemy, as well as certifications like AWS certifications and Kubernetes certifications or documentation like MLRun documentation. In addition, he highly recommends having annual and quarterly learning goals, otherwise learning might be put on the back burner, as it will often become a lower priority.

3. Follow Future Trends

It has almost become a cliche to say we live in an ever-changing world. But with new technologies and capabilities becoming available on a monthly basis, it is important to keep up with the most significant machine learning trends, to keep products future-proof.

Here are the main trends Gift expects to see in the upcoming one to two years:

• NFSOps: Network File Systems, Windows storage and FSX systems are making a comeback. Using an EFS as a source of truth and mounting a cloud based development system on top of it can create a feedback loop and sync code into the NFS and to 1000s of deep learning nodes and containers. This simplifies workloads and provides access, instead of having to manage Hadoop through a cluster, for example.
• Kubernetes and Kubeflow: Most companies will probably be using a technology that is influenced by Kuberenetes or Kubeflow, or an abstraction that manages it, like Amazon EKS. (If they aren’t already doing so).
• Edge ML: Instead of relying only on APIs or batch-based jobs, deploying to an edge-based device can help solve some technological problems. TensorFlow and pretrained models may come in handy here.
• ESG (Environmental, Social, Governance): Implementing ESG means being conscious about the environment. This movement is gaining traction, especially by the new generation. Companies that are not embracing ESG will be avoided. How can you adopt ESG? You can make sure to verify which models really need to be retrained (and which training is just a waste of resources), implement ethical AI, and more.
• Pre-trained Models and AutoML: Reusing models, even ones that were trained by others, will help save time and resources. If you don’t like the sound of this, think about pizza. Eating a pizza doesn’t require you to grind wheat to make flour. Instead, you can order one, warm up a frozen pizza or make dough and put cheese and sauce on top of it. Just like pizza, data scientists can use models through sentiment analysis APIs or AutoML, or they can train models themselves.
• Model Portability: Porting models to different languages, like ONNX or Apple’s CoreML models. This enables working on models across departments, companies and geographies.
• Kaizen ML: Automating all ML processes to ensure continuous improvement in model creation and performance.

We believe these trends will make a significant impact on machine learning in the upcoming years.

4. Adopt a Production-First Mindset

Adopting a production-first mindset means starting out by designing the continuous operational pipeline for bringing data to production, rather than with research and model training. This enables scalability and ensures the pipeline can provide business value while answering MLOps enterprise needs. 

The following four key components can help with adopting such a mind-set, and be used for data generation from live sources, continuous deployments, and more:

1. Feature Store - A feature store enables anyone on the ML team to grab data from a variety of real-time or offline sources for building or retraining in an automated fashion. 
2. Real-time Serving Pipeline - Using serverless engines that build features automatically and bring them into a training pipeline, including  the API handling, data preparation/enrichment, model serving, ensembles, driving and measuring actions, etc. This pipeline can be integrated with CI, as well as Git and monitoring systems.
3. Monitoring and Re-training - Drift detection and automated retraining. Monitoring provides a feedback loop for exploring production data,, alerting on anomalies or data quality issues, triggering re-training jobs, measuring business impact, etc.
4. CI/CD for ML - Pipelines for automated machine learning, model deployment, training, version tracking, etc. These pipelines can be plugged into CI/CD solutions like KubeFlow pipelines, GitHub Actions, Jenkins, etc.

You can watch a demo of these components in the webinar, available on-demand here.

Conclusion

By adopting these best practices: 1) Picking the right technology partners, 2) Leveraging online learning and certifications, 3) Following future trends and 4) Adopting a production-first mindset, data science professionals and leaders can ensure their work has a clear business ROI. This provides value to their work and can help make their company, and their findings, successful and widely-adopted.