They say half of life is just turning up.
I would say the other half is not falling into deadly traps!
The same goes for data: the path to the top of data mountain is fraught with traps and pitfalls of all shapes and sizes. Most of the battle is avoiding epic fails, rather than having to be spectacularly successful in some way.
In this blog, I’ll look at some of the deadly traps that will ensnare your data dreams and crush them into dust!
This is the trap of failing to integrate your data strategy into the wider goals and objectives of your organisation.
A data strategy determines what data you’re capturing, how and for what purpose.
If it’s not intimately woven into the wider tapestry of the business then your data efforts, instead of being a focused laser, will be a wild firework display that makes a lot of noise but achieves very little.
You won’t catch any bears. Just annoy them with sparklers.
Include business folk from the outset on your data strategy team, ensuring that business objectives form the north star to which all data efforts are inextricably oriented.
All projects (and the incentives/metrics that measure their success) must be explicitly linked to a business goal.
The data pyramid scheme is when you try to build advanced data analytics, ML and AI without first building a solid ‘pyramid’ or foundation of basic data capabilities.
Most companies are not ready for the likes of ML and AI, so when they try to jump straight in without laying the groundwork first, they get impaled on the spiky tip of the data pyramid!
Ouch.
How to Avoid the Trap
You have to lay rock-solid foundations before attempting any advanced data shenanigans.
This includes people (the right skills organised in the right way in the right culture), processes (impeccable data governance, operating model and metrics) and technology (infrastructure and tools that are in the Goldilocks Zone [i.e. just right]).
This trap is reserved for those who overlook the importance of managing the quality, availability and security of your data.
Low-quality data that is poorly organised, hard to access and that no one takes responsibility for...inevitably leads to poor business decisions, regardless of how sophisticated the rest of your data platform is.
Your data program ends up being like a million dollars sitting in a bank account that you’ve forgotten the PIN code for.
Deploy a data governance team and appoint one (or more) data steward(s). These will have responsibility for creating and maintaining standards and policies for governing data across the organisation.
Monolithic architectures are too tightly-coupled and interdependent: they create the illusion of providing everything you might need...but do not!
Because monolithic architectures are so tightly coupled, when you want to change one little corner, you have to redeploy the whole thing. It’s like having to repaint your entire house every time you want to give the front door a fresh coat.
In the end, your data platform will not be able to scale and/or change at the pace required for real-time data analytics.
Embrace distributed approaches to data architecture: cloud-based microservices or the data mesh. I previously explored how data mesh will unlock the promised value of data in the enterprise.
Why not try to go all-out and achieve all your data goals at once with one giant plan, which you have never tried before, executed all in one go?
It’s so tempting and what could possibly go wrong?
The attempt at boiling the ocean leaves no room for flexibility or agility in how you approach your data strategy.
Getting your entire data strategy right upfront is like trying to score a hole-in-one on the world’s gnarliest golf course.
Upfront investments and expectations are too high and, once it has begun, the sunk costs are such that the pressure to proceed with the strategy is incredible, even if it obviously isn’t working.
Instead of going for the hole-in-one, take each shot on its own, reassessing after each one. Start small, experiment and see if you can create some small business value. With this in hand you have a mandate to scale your operation, carefully and gradually.
Enterprise data platforms are nearly always centralised, which becomes a jumbled pit of snakes when you start to scale this central platform across many thousands of employees, teams and lines of business.
As more and more disparate datasets accumulate it becomes harder and harder for someone to find what they need and make sense of it. At the same time, response times slow as more and more people pile into the same platform.
Move beyond centralised data lakes and warehouses to taking a data mesh approach: organising data in a distributed way by business domain, not by pipeline stage.
This scam arises when data scientists put too much effort into their architecture and algorithms and generally being very clever, but neglect the needs of the end user, who struggles to get access to the data they need!
This is like buying a winning lottery ticket, but not being able to find it. You’ve got the potential, but the final delivery is missing!
Bring product thinking into the equation. Start treating your data as a product, the success of which is defined by how happy it makes its customers (in this case, your internal teams).
You fall into the spike pit when you assume that you will be able to force new tools and new ways of working onto your employees without resistance.
Without bringing your folks on the transformation journey with you, expect substantial scepticism and resistance from key individuals, causing chronic delays and headaches.
Make education and communication a central pillar of your data strategy, equal in importance to the technical aspects.
Incentives and metrics are meant to motivate people to move in the right direction and then to track progress in that direction. When they don’t do this well, you are camouflaging what’s actually going on under a veneer of numbers.
With misaligned incentives you have individuals and teams working against each other. Meaningless metrics mean you’re tracking things that are not valuable for achieving your business objectives.
When you combine these, you end up with lots of conflict and wasted time that can be camouflaged as success and progress!
Ensure that incentives and metrics are aligned with business KPIs and that teams are accountable for these KPIs from end-to-end.
This mirage appears when business leaders lack data literacy and are unable to weave a story that articulates the ‘why’ of data, or to derive a clear direction from the insights that their own data present them.
Your leaders will be unable to take advantage of the clues that your data provides as to which action is best, instead returning to making decisions based on guesswork, rather than being data-driven.
Leadership teams must be equipped to make full use of the slew of data insights that will be generated by a well-functioning data strategy: education and inspiration are key!
You make a deal with the devil when you try to operationalise an epic data strategy...using the wrong operating model!
That means having team structures, accountability dynamics, processes and ways of working that rub against, rather than flow with, the way data needs to move through your organisation.
An inappropriate operating model creates a thousand small bottlenecks, pressures, delays, disconnects, misunderstandings and misalignments that add up to a data capability that is too slow, cannot be scaled and that is disconnected from the business.
A modern operating model must reflect the structure of your infrastructure as well as your teams: all should be synchronised to work together and minimise misalignments, delays and blocks. Typically this means building modular infrastructure and teams based in pods or squads that are aligned to specific products.
You cage yourself when you break your teams into siloes of similar specialists that are disconnected from other teams or areas of the business.
The chain of accountability from one end of your data lifecycle to the other is broken. The result is that teams focus on their own tasks at the expense of the wider view and are disconnected from higher business goals and objectives.
Build cross-functional teams of experts from across the business that have everything they need to shepherd a particular product or feature from start to finish.
I’ve previously explored the importance of the data mesh, machine learning, artificial intelligence and now avoiding major traps to modern data excellence.
What are the chances of there being a company that deeply understands what it takes to make these advanced and interdependent data strategies a reality, while avoiding the largest and deadliest traps?
