Implement a Data Mesh approach to data architecture to make data accessible, interconnected and valuable for data science teams across the organization
One of the biggest challenges in building modern data products is working with live data from different domains within the enterprise, and dealing with its processing, scale, governance and security. With the constant change of an enterprise’s data landscape, the proliferation of data sources, and the diversity of use case requirements, it is a challenge for organizations to nimbly roll out AI services to meet business needs.
Much like the transition from monolithic apps to an assembly of microservices, the concept of a data mesh approach attempts to address the limitations of monolithic or central data architectures by turning to a distributed data architecture where each domains handles the required data processing, governance and connectivity to other domains or applications.
The Iguazio feature store and MLRun open source framework are some of the first solutions that support data mesh architecture principles, where data assets are broken into domains (called feature sets within Iguazio) and each domain at the helm of connecting, transforming, cataloging, governing, and serving of its data. The feature store further extends the data mesh architecture with feature vectors, which join data automatically from multiple domains in real-time or batch and address the seamless integration with ML applications.
The feature store is implemented using a self-served serverless and microservices architecture (over Kubernetes), providing elastic scaling, self-healing, costs reduction, and continuous operation.
Easily engineer features and share them in an easily consumable way for any purpose downstream.
Leverage the data mesh concept to quickly build data products across domains. With the Iguazio feature store, data is organized around domains, and infrastructure is abstracted away.
Share, search and collaborate on features in a centralized and versioned catalog, to quickly build data products across domains. Seamlessly integrate ML and analytics tools.
Manage users and policies in a multi-layered data authorization scheme for IT administrators, allowing data scientists and engineers to work in a flexible ecosystem without worrying about security.
Learn how to implement the data mesh approach to data architecture with the Iguazio AI Platform
