The arrival of the data mesh paradigm has brought to light the challenges faced by traditional approaches to analytical data use cases.
For example, many organisations struggle with the poor scalability of monolithic data platforms, both technical and organisational. These platforms are often implemented as a technology-first exercise that end up creating multiple bottlenecks on different levels, compromising delivery and data accessibility.
Ultimately, they fail to deliver the anticipated value for many of today’s data-driven organisations.
By contrast, data mesh introduces a decentralised, data-centric approach that addresses these challenges at the core.
The canonical concept of the data mesh focuses purely on analytical use cases.
And by focusing only on analytics, we miss the opportunity to tackle some deeper issues that are connected to analytics.
Because here’s the thing: analytical use cases do not exist in isolation.
If we take these use cases out of their broader context (data and software) we could be missing out on solving the real problems that hamstring scalable data utilisation in many other areas of the business.
In this blog, I want to make an argument for broadening the use case horizon of the data mesh.
There are a few key reasons:
We absolutely need to avoid turning analytics into its own silo (even if it is a nicer one!).
Instead, we have to consider the broader data and software picture to be able to design and implement far-reaching and durable solutions that will cover both ends of the data spectrum: operational and analytical, and everything in between.
The secret sauce here is the data itself: fundamentally, what all data-centric use cases have in common is the need to access reliable, trustworthy and timely data adapted to their specific requirements. And that is what needs to be brought within the purview of the data mesh.
Through this lens, data mesh can be seen as a cohesive umbrella and a set of guiding principles for a data-centric transformation:
These principles can be applied to any data source or use case type. None of these are exclusive to analytical data. For example, you should be able to capture a “real time” data stream in a data catalogue, just as you would do the same for a data set sitting in a relational database or a lakehouse.
These principles are not even unique to data. We can see data mesh in its historical context: as a natural step forward to bring to data principles and ways of working that have been proven to be successful in other aspects of IT.
To (over)simplify a bit, we could think of a data mesh as the culmination of a number of successive and overlapping movements sharing the same ethos as data mesh:
It is great to see that the technical and conceptual maturity are catching up to allow us to put data where it deserves: at the centre of any meaningful evolution or transformation.
As we saw, data use cases fall on a broad spectrum of the analytical/operational dimensions but they all share the necessity to access usable data. The issues affecting these use cases are common and far reaching.
Data mesh is based on a set of powerful principles that will allow data-centric organisations to come to grips with the real problems that are impacting their ability to innovate at scale. Dealing with these problems will yield more meaningful benefits to the full spectrum of data use cases.
These principles have not emerged in a vacuum. They are rather the natural extension of previous movements that dealt with similar challenges in other areas of IT: infrastructure, applications, teams and products. The data itself has taken a sort of a back seat up until now. We now have the tools and the understanding to bring data back to the centre and to move into the future by revisiting the past!
Interested in seeing our latest blogs as soon as they get released? Sign up for our newsletter using the form below, and also follow us on LinkedIn.