What is Data Mesh, and Do I Need It for My Organization?

Data mesh emerged out of the frustration of leaders and practitioners who were fed up with the data strategies that pervaded the last decade. For many large or complex organizations, the ever-common strategy of central data platforms, central data integration, central data governance, and central analytics teams have not kept pace with the demand for information. In an area of instantly accessible information, waiting weeks or months for answers to questions is unacceptable.

The data mesh is meant to answer the call for a ‘better way’ by:

• Enabling decentralization and localizing change to business domains.
• Delivering a data solution with the customer in mind.
• Empowering cross-functional teams to share data.
• Providing a data governance operating model with federated decision-making and accountability.

In this blog, we cover:

• Is Data Mesh Right for You? ↵
• What is Data Mesh? ↵
• What are Data Mesh Principles? ↵
• When Would Data Mesh Make Sense (or not) for My Organization? ↵
• Can I Apply Bits and Pieces of Data Mesh Until I Am Ready to Go? ↵
• Is There a Single Technology You Can Use to Implement Data Mesh Quickly Today? ↵

Is Data Mesh Right for You?

Here are five key questions about data mesh that will help you understand what it is and who should adopt it.

1. What is data mesh?

According to the founder of data mesh Zhamak Dehghani, “Data mesh is a decentralized sociotechnical approach to share, access, and manage analytical data in complex and large-scale environments — within or across organizations.”

There are two key concepts to unpack in this definition:

• Sociotechnical means that data mesh is a strategy that fundamentally is dependent on people and technology. Data mesh itself is not a technology, architecture, operating model, or data governance model. It is all these things — and probably more.

If you plan to adopt data mesh, your data strategy must address human interaction in central focus. Because sharing data among teams is at the core of data mesh, the main challenges you will face will more likely be related to human behavior than technical competency.

• A decentralized approach is key to data mesh, which rejects the idea that a central approach to data and analytics is feasible and scalable within complex organizations. The assumption of data mesh is that your organization has passed the critical inflection point for complexity, size, scale, or demand, and a centralized operating model will no longer work.

What is data mesh and how does it address fears and issues with decentralization?

2. What are data mesh principles? 

Data mesh was founded on four key principles — domain ownership, data as a product, self-serve platform, and federated computational governance.

There are four key principles to data mesh—domain ownership, data as a product, self-serve platform, and federated computational governance.

• Domain ownership enables the process of decentralization by aligning technical and business teams. A decentralized domain-oriented approach optimally places the responsibility of turning data into information onto the domain that created the data in the first place. Each domain is accountable and responsible for data quality and completeness of what is shared with the organization.
• Data as a product means applying product development thinking to data solutions. In this model, company data is viewed as a product and the data team’s role is to provide that data to the company in ways that facilitate good decision-making and meet company objectives. Data as a product also necessitates that products be valuable on their own to reduce interdomain dependencies that slow down progress of solution development.
• Self-serve data platform allows for a framework for everyone within the organization to discover and use data products without having to develop particular point solutions or data pipelines to share information. This empowers cross-functional teams to share data seamlessly with one another.
• Federated computational governance clearly defines parameters for how data can be shared and used — especially accounting for legal considerations, compliance mandates, and security protocols. It also enables correlation of independent yet interoperable domains, as well as codifying and automating policies — especially in avoidance of manual processes.

3. When would data mesh make sense (or not) for my organization?

Dehghani describes eight different criteria areas to assess whether an organization is ready to adopt data mesh currently including organizational complexity, data-oriented strategy, executive support, data technology at core, early adopter, modern engineering, domain-oriented organization, and long-term commitment.

While all these areas are important, I want to focus on the areas where we see organizations are most likely to disqualify themselves:

• Organizational Complexity: Many organizations today lack the scale or complexity to benefit from data mesh. They are better positioned to benefit from a centralized approach using a data warehouse or a data lakehouse to deliver value quickly to their organization.We see many organizations that consider themselves complicated, while in reality their problems are very solvable with well-adopted and simpler data strategies.Data mesh could introduce unnecessary complexity that could inhibit growth in the short-term while they work to scale up their teams, processes, and technologies. Believe it or not, a lot of the world still runs their core business operations day-to-day in spreadsheets. Data mesh would represent a dramatic shift in maturity that can be unattainable given their current teams and talents.If data sources and use cases can be easily obtained and well-defined by a central team, then a centralized model makes much more sense to adopt.

Data mesh is a valid option only when there is an inflection point of complexity that warrants a shift in thinking. Photo credit: “Data Mesh: Delivering Data-Driven Value at Scale”

• Executive Support: Having executive buy-in for any initiative is critical. Data mesh is no different — it cannot be forced upon the organization purely from the technical team. It requires that the relationship between business functions and technical teams be fundamentally redefined.That sort of change will not be successful without executive investment to stay disciplined and focused on data mesh as a strategic transformation to the organization.
• Domain-Oriented Organization: Dehghani says that data mesh assumes that the ideal organization’s domains already have dedicated technical teams that can build and support digital assets and products.If your organization is built upon a centralized shared model for IT, you may not have enough people or leadership to effectively implement data mesh today. The risk here is only going halfway — to partially orient teams to one or more domains while still maintaining centralized leadership and governance. This approach spreads teams far too thin and maintains old ways of thinking.

4. If I’m not ready, can I apply bits and pieces of data mesh until I am ready to go?

No and yes.

Looking at the sort of long-term wholesale organizational strategy shift required to fully adopt data mesh, it is no wonder that we see organizations dabbling with partial strategies to trial data mesh. But there is great risk in this way of thinking, especially if you lack the fundamental components necessary to avoid well-known pitfalls with decentralized data delivery models.

For example, if you fully decentralize data teams without having proper governance and expectations for the products they are creating, you’re going to create a data mess — not a data mesh.

However, there are parts of data mesh that are great to adopt as a part of simpler well-defined data strategy. For example, having high end-user empathy by thinking about data as a product is a principle that you can adopt with central data teams to improve solutions that you’re delivering today, even if you’re not ready for a full adoption of a data mesh as an organization.

You can start slowly by developing clear expectations for what a quality data product looks like today so that when you reach a plateau of productivity with a central model, you’ll be more ready as an organization to decentralize and domain-orient your data products with a data mesh approach.

5. Is there a single technology you can use to implement data mesh quickly today?

Not in my opinion. Creating a data mesh is not primarily a technical problem, and if you hear it being reported as such by a technology vendor, it is best to walk away.

Now, there are many technical components of a data mesh that you can build using established technologies today, but it is far more important to address organizationally how you think about data, how you steward data, how you gather domain requirements, and how to discover and share reliable information. All these concepts cannot be solved by a technology alone.

As the market sees the value in a decentralized sociotechnical approach toward analytical data, better and more holistic solutions may appear that address all demands of creating a data mesh, but we’re not quite there.

Our advice: to start on your data mesh journey, fully assess your organization’s need and readiness to adopt a data mesh.