I’ve been privileged over the past few months to have the chance to contribute to work being done by the Leaders Data Group (www.dataleaders.org) looking at ways to move the discussion forward on data management fundamentals in the face of emerging trends like AI and Machine Learning. The motivator for this work is simple: the same mistakes are being made again as we have lived through the last iteration of shifting paradigms in information management with the allure of new technology and software tools, and the one before that, and the one before that.
This is something I’ve taken to calling the “lure of the Shiny.” As technology innovators develop new ways of working with information, technology implementers want to adopt and implement these new technologies, which always promise significant benefits and upsides, and the business stakeholders invariably buy into the promised vision. I’ve seen this cycle repeat in CRM, in ERP, in “Big Data” (before it took a stats class and became AI), and in various iterations of magic bullet software for regulatory and governance ills.
The “lure of the Shiny” tells us that if we buy the latest widget or doodad or slick technology we will fix the ills of the organization in a jiffy, unleashing the power and potential within that is just shackled up in a back room somewhere waiting to break free. Only this week I spoke with a client who has a new CIO who is looking to build a Data Warehouse in the Cloud while the CFO is looking to move away from the current term “EMP” (Extract, Manipulate, PowerPoint) reporting processes to a slicker on-demand Business Intelligence capability. All of these are wonderful and laudable objectives with potentially useful benefits to the organization. However, neither of them addresses the fundamental information management challenges of the organization and, unless the questions about those fundamental challenges are asked and answered, both of those initiatives are doomed to failure.
Of course, EMP-based reporting shares the potential of its electrical namesake in that it can quite quickly shut down the electronic processes of the organization as the reporting takes time. The extraction of data, the manipulation and massaging of data in Excel, and the packaging of the analysis in a management reporting pack takes human effort. With the client I spoke to, they were estimating one report took 2.5 person-days per month to run. Crunching their internal activity-based costing numbers, that one report is costing them €13,500 per year to run. So, obviously replacing this cumbersome process with a self-service BI process would make a lot of sense. Because the organization has 20 of those reports, anything south of €270,000 investment has a one-year payback period.
It all makes sense.
But it is all doomed to failure if the organization has been dazzled by the “lure of the Shiny”. Because, if the organization hasn’t considered what the transformations and manipulations are that are done in the EMP reporting process, if they haven’t considered whether the multitude of reports are simply asking the same questions in different ways, and if they haven’t considered the potential data quality issues that will exist in the information chain from the source systems to the reporting system, then the removal of the human work-arounds that correct errors and perform reconciliations in that manual process will just result in poor quality data being delivered to front-line decision makers and knowledge workers.
If the organization hasn’t invested in educating these knowledge workers, who historically have been presented a static view of data, along with the meaning, purpose, and relationships in data, on the business rules within the data, and on the basics of how to frame and construct queries, then the organization will have replaced their EMP with a loaded gun.
Fundamentally, the “lure of the Shiny” can best be understood by an analogy I drew a few years ago on a conference panel. The promise of new information management technology is like the promise of the home gym equipment on the late-night shopping channel. The advert and demonstration tell you that you too can develop a lean fit physique (if you send six instalments of $89.95 – callers are waiting). The product might even be endorsed by a famous fit and lean person. However, the advert fails to tell you (explicitly) that the transformation is ultimately about some hard work and basic physical rules and that the shiny equipment can help, but unless you are talking about and dealing with those fundamentals of diet and lifestyle any improvement will be fleeting.
So, how do we fight back against “the lure of the Shiny”?
Well, the essential step is to take a long hard look at the strengths and weaknesses of our organizations regarding data and how we manage it. Only by being rigorously honest and objective about how we manage data quality, how we address data governance, and how well we manage metadata (to name but a few) can we begin to map realistic roadmaps for business capability development in any of the “shiny” information management disciplines.
And it is essential that this conversation at least includes business stakeholders as equal partners and participants, if not as the actual leaders of the discussion. After all, if the CFO really wants responsive self-service BI, perhaps they might want to understand how a cloud-based Data Warehouse might help. They also should know what organization, process, and governance changes will be needed, in addition to writing the PO for the software and the systems integrators, to deliver a step change in business information capability that will deliver sustainable value.
Looking back in the blog archive on Castlebridge.ie I find that this isn’t the first time I’ve thought about and written about this topic. Here’s that blog post from 2015 https://castlebridge.ie/2015/07/30/fear-and-loathing-in-the-data/.
The Leaders’ Data Group will be releasing its first work package shortly. It’s aimed at helping organization start to have these types of conversations in the context of AI and Machine Learning to make sure your organization. Check out dataleaders.org for more information over the coming weeks.
