MLOps for Gen AI in the Enterprise

Generative AI holds great promise for organizations, sparking innovation while providing ROI and economic value. According to a new research report by Quantum Black, AI by McKinsey, titled “The economic potential of generative AI” there is up to $4.4 trillion in annual global productivity to gain. 75% of this impact is focused on marketing and sales, software engineering, customer operations and product and R&D.

However, it is important to acknowledge that securely and responsibly implementing these large models in user-facing production applications poses a new and intricate challenge in MLOps.

In this insightful session, Yaron Haviv, CTO Iguazio was joined by Nayur Khan, Partner, QuantumBlack, AI by McKinsey and Mara Pometti​, Associate Design Director, McKinsey & Company, to discuss how enterprises can effectively incorporate GenAI now in live business applications.

Webinar Highlights:

According to Quantum Black, AI by McKinsey, there are four use case archetypes that are being developed first: the virtual expert, customer engagement, coding and content generation.

In addition, there are three main ways organizations are adopting generative AI today. First, the takers, who use APIs of commercial or open source LLMs. Second, the shapers, who fine-tune pre-trained LLMs with additional data. Finally, the makers, who create and train models from scratch.

In the session, we dive into a “Taker” example, showcasing what it requires from organizations, engineering teams and data professionals. For example, to get started organizations need to support app development according to the 12-factor app guidelines, an environment that follows proper DevOps and DevSecOps practices, infrastructure, and more. There are also model selection considerations, LLMOps challenges to take into account, guardrails against injection, bias considerations, and more.

However, GenAI also raises concerns. The use of GenAI needs to be responsible. In the wrong hands, the results could be disastrous. This means building guardrails based on a human-centered approach.

In the webinar, we demonstrate an example of implementing a human-centered approach to GenAI through a customer engagement solution. The app, which uses GenAI to interact with an example customer, Lucila Esteban, a 54 years old singer.

With GenAI, the bank transforms large amounts of customer data into short text summaries, automates campaigns tailored to the customer including choosing the right engagement channels.

The design of the customer engagement solution was based on human-centered practices, while also embedding Responsible AI into the development. This was done to ensure each message sent to customers is reliable and trustworthy and does not contain toxicity, harmfulness, bias and stereotyping, regulatory risk, or hallucinations,

Finally, we delve into Generative AI and MLOps best practices that can simplify and automate deployment foundation models.

There are two main techniques for customizing LLMs. Prompt engineering, which generates a prompt along with data enrichment or examples, and fine tuning and transfer learning, which uses an existing pre-trained model and further training it with an application specific dataset. 

GenAI models can be complex to operationalize due the large scale data that needs processing and training, the difficulties when it comes to continuous tuning and redeployment, the challenges of achieving end-to-end observability and governance, and more.

In the webinar we show how to overcome these challenges with a five-step process for building and deploying foundation models: data creation, data curation, training, adaptation and deployment. We also show how to build and deploy a GenAI training and validation pipeline and a serving pipeline. There is also a demo of how to use open source MLRun for building these pipelines.

Watch the MLOps for Gen AI in the Enterprise webinar to watch the entire discussion.