Automating Machine Learning Pipelines on Azure and Azure Stack

Ever wonder if it’s possible to train machine learning (ML) models with regulated data which can’t be sent to the cloud? Has your edge solution gathered so much data that it just doesn’t make sense to send it all to the cloud? Iguazio brings the cloud’s intelligence to the edge enabling you to perform analytics locally and send aggregated data to the cloud.

Iguazio’s partnership with Microsoft creates new possibilities for Azure and Azure Stack customers to develop end-to-end ML-based applications which may reside in the cloud, at the edge or across a hybrid deployment. Iguazio augments the capabilities of Azure and Azure Stack by providing:

• A self-service framework for ML tools based on leading open source ML projects
• A high performance and unified data fabric
• Serverless automation with Nuclio

A Complete Machine Learning Pipeline Running in the Cloud, at the Edge or as a Hybrid Deployment

With Iguazio’s Nuclio Serverless Functions, users collect data from various sources and types. Nuclio provides fast and secure access to real-time and historical data at scale, including event-driven streaming, time series, NoSQL, SQL and files.

Data scientists explore and access data using a Jupyter notebook and work with popular frameworks such as Spark, Presto and Pandas. Users store and access data with different formats, including NoSQL, time series, stream data and files, while leveraging different APIs to access and manipulate the data, all from a collaborative development environment running at the cloud and edge.

Iguazio’s open Python environment with built-in ML libraries like Scikit Learn, NumPy, Pytorch and TensorFlow over Kubernetes, enables users to build and train models easily. When a model is ready, users validate it against a real-time production-like dataset on a distributed cluster, leveraging frameworks such as Dask, Spark, Horovod and Tensorflow with GPU support.

Users deploy models from Jupyter notebook to production with just a few clicks and in a reproducible way. The code is deployed as a serverless function with all of its relevant configurations and is immediately ready to run at the edge, cloud or both.

Iguazio provides workflow automation as a service and includes KubeFlow, the leading tool in the industry for ML pipeline management. This enables users to streamline the process of experimentation and inferencing.

Customer Demos Presented This Week at Microsoft Inspire 2019:

REAL-TIME LOCATION-BASED RECOMMENDATIONS

A leading telco uses Iguazio to recommend products to customers in real-time based on historical behavior, profile and geo location. The platform collects real-time transaction and location data enriched with open and regulated data. Data is then analyzed with a location/topic-based recommendation engine which sends matching product coupons via text messages. Users view real-time recommendations and predictions on live dashboards.

REAL-TIME STOCK PERFORMANCE AND SENTIMENT ANALYSIS

Global banks deploy Iguazio to predict and avoid latencies in their electronic trading platforms. The platform performs real-time tweet sentiment analysis and tick feed analysis using Jupyter notebook, Spark and time series data. Real-time results and predictions are served on Grafana dashboards.

INVESTMENT RECOMMENDATIONS

A leading financial services company analyzes customer transactions and proactively sends recommendations. Iguazio provides a secure and shared environment with the required building blocks for a seamless customer experience. It simplifies the process from training to production, reducing dependencies on data engineering and ensuring that the production and training environments are the same.

REAL-TIME INFRASTRUCTURE MANAGEMENT

A global telco uses Iguazio to predict network outages and avoid them in real-time. The platform processes high message throughput of time-series data from multiple
streams correlated with historical data, to generate real-time ML-based outage predictions, root cause analysis and pre-programmed triggers for preemptive fixes.
Insights are served via interactive dashboards, alerts and automated actions.

THE FIRST INTEGRATION OF KUBEFLOW WITH AZURE STACK

Iguazio’s managed KubeFlow enables Azure customers to manage experiments, runs and artifacts and build workflows using code or reusable components. With Iguazio, users gain seamless data access and parallelism, authentication, RBAC and data security, distributed training and GPU acceleration as well as execution, data tracking and versioning.

Meet with Iguazio's team all week at the Edge Infrastructure area in the Inspire expo hall to see one of the above demos. Iguazio is also available on the Azure and Azure Stack Marketplace.