How HR Tech Company Sense Scaled their ML Operations using Iguazio

Sense is a talent engagement company whose platform improves the recruitment processes with automation, AI and personalization. Since AI is a central pillar of their value offering, Sense has invested heavily in a robust engineering organization including a large number of data and AI professionals. This includes a data team, an analytics team, DevOps, AI/ML, and a data science team. The AI/Ml team is made up of ML engineers, data scientists and backend product engineers.

The Challenge

Like many organizations, the AI/ML team at Sense was finding it challenging to scale its ML operations. This was mainly due to three factors:

• Complexity when managing multiple projects and experiments - Sense had to determine the best strategy for conducting and controlling all their projects and versions, at scale.
• The need for speed while supporting developer efficiency - Sense needed to ensure fast time-to-market while managing resources efficiently.
• Establishing a deployment and monitoring strategy - Sense needed to create a sound deployment and monitoring strategy in a cost-effective and straightforward manner.

The Solution

Sense chose Iguazio as their MLOps solution. Iguazio is an essential component in Sense’s MLOps and DataOps architecture, acting as the ML training and serving component of the pipeline. With Iguazio, Sense’s ML team members can pull data, analyze it, train and run experiments, making the process automated, scalable and cost-effective.

“With Iguazio, data scientists and ML engineers start having superpowers.” Gennaro Frazzingaro, Head of AI/ML at Sense.

Iguazio: A Key Component in Sense’s AI/ML and Data Stack

Sense’s MLOps and DataOps platforms were developed to answer three main objectives:

1. Making data accessible so it can easily be used for building ML models.
2. Enabling quick iterations over feedback.
3. Enabling quick experimentation.

All teams collaborated to develop these platforms and to determine which technology and interfaces to use.

The system’s architecture ensures the data flows through the different systems effectively.

First, the data lake is fed from a number of data sources. These include conversational data, ATS Data and more. These data pipelines are centralized into one Snowflake instance, which is the central repository for any of Sense’s data needs.

Sense onboarded Iguazio as an MLOps solution for the ML training and serving component of the pipeline. Iguazio integrates with Snowflake and with additional systems in the stack. One of these systems is Label Studio, which is used to accelerate labeling and data annotation needs.

The teams at Sense also use an ML runtime platform for A/B testing, deployment and monitoring and Sigma for dashboarding and deriving insights.

With Jupyter notebooks, Sense teams analyze data, reinforce models, experiment with use cases and solutions, and more. For example, they can determine which incremental improvements they need to deliver to the model to adapt to real world language and which use cases to prioritize.

In addition to a robust AI/ML and data stack, Sense also established a sound training, evaluation and experimentation process. By clearly defining processes that align with industry best practices, they can ensure the efficiency of the ML project.

At the start of each project, Sense sets two priorities:

1. A thorough evaluation strategy that derives progress and discards any unsuccessful experiments quickly.
2. Never compromising on agility. This includes reinforcement learning through feedback, which boosts ML engineers’ and data scientists’ productivity. At Sense, any ML engineer or data scientist can train models on remote GPUs and manage the full stack with little or no involvement from DevOps.

The Results: Why Sense Chose Iguazio

Before choosing Iguazio, Sense evaluated multiple alternatives. Sense picked Iguazio over competitors because Iguazio provides a solution to their requirements, like:

• Comprehensiveness: Iguazio is an all-in-one solution
• Agility: Iguazio helps Sense iterate quickly
• Easy onboarding: Newcomers to MLOps can quickly ramp up with Iguazio
• Quick deployment: Installation on the AWS cluster is quick and easy.
• Scalability: Iguazio leverages Kubernetes, which simplifies traffic-based scaling.
• Cost-effectiveness: Sense was able to find the ideal AWS cost and resource allocation balance. Sense uses GPU-bound instances to speed up the training pipeline and their staging environments and less GPU memory for the serving pipeline interaction environment.
• Low Latency: With Iguazio, P99 is below 100 milliseconds in Production at peak. Sense’s Chapel backend proxies to Iguazio, which serves the model in production on every single user message.
• Flexibility: Open source MLRun (built and maintained by Iguazio) provides devs and scientists with a way to manage the full stack using Python. For example, Python can be used to schedule how they want to run their jobs.
• IaC: In Iguazio, the infrastructure is fully managed as code, and it's managed in GitHub. This means it goes from a code review, alongside the numerous advantages of having infrastructure as code.
• Advanced Monitoring: Iguazio provides a wide scope of monitoring capabilities. This starts with out-of-the-box monitoring for all elementary needs up to enabling the easy building of dashboards on top of the standard ones.
• Iguazio’s support: “Iguazio’s support since the start has been phenomenal. They've been partners of ours every step of the way.” Gennaro Frazzingaro, Head of AI/ML at Sense.

How Sense uses Iguazio

Sense uses Iguazio heavily for a variety of different projects. For example:

• Building a comprehensive and sophisticated AI chatbot.
• For closed-source models like OpenAI and Anthropic.
• Fine-tuning open source alert apps
• Experimenting with various deep learning techniques, which optimize Sense’s products’ ability to effectively match candidates to jobs or jobs to candidates

In addition, Sense plans to use Iguazio for a future product called Sense co-pilot. The co-pilot will help recruiters work more efficiently by predicting what they should be doing next in the platform.

“I would truly recommend using Iguazio.” Gennaro Frazzingaro, Head of AI/ML at Sense.