Published in TDAN.com July 2006
After 16 years in the data quality software business, having worked with countless Fortune 500 firms of every stripe, I felt that I had a pretty good handle on the trials and tribulations of
managing customer data. I had seen all manner of data management “challenges” – from poor quality to bad design to faulty processes. I came to believe, after a few years, that these complex
issues must be an unwelcome yet unavoidable byproduct of running a multi-billion dollar international conglomerate – there is just so much data generated everyday that getting effective
control of it is nearly impossible. When I began to consult with smaller firms as an independent, I reasoned that these same issues must be facing the owners and senior executives of
small-to-midsized businesses (SMBs). To some extent that is true, but it doesn’t tell the whole story.
Too often, the approaches to solving some aberrant condition impacting customer data involve an ill-conceived or poorly executed initiative. The great disappointment comes when the client realizes
(usually some months after the “solution” has been implemented) that the quality of the data, while improved, isn’t meeting expectations. It isn’t for lack of management attention or a failure
to comprehend the true costs of poorly managed customer information. Multiple millions of dollars (literally) are invested in software and professional services from experienced and erstwhile
vendors. Yet the problems persist, regardless of senior management commitment to implement a solution at whatever cost.
The common denominator in these scenarios is very simply this: despite all the planning, preparation and exertion lavished on technical answers to the customer data quality conundrum, the primary
culprit is and will always be the human factor. People generate the bulk of the customer data overwhelming businesses today – large or small. Whether through the sales channel, a call center,
direct service channels, marketing initiatives or customer supplied input (i.e., internet forms-based collection); human beings create data related to customers. Certainly there are systems in
place that may be able to prevent the most commonplace mistakes from occurring (e.g., software tools such as callable API versions of address validation routines will ensure that addresses are
mail-able and zip codes are correct). Training and field level validation may reduce the number of incorrectly populated columns in a database – such as filling a social security number field
with 999-99-9999. When all is said and done, however, a human being will always be capable of finding the path of least resistance to get a job done. If the job happens to involve customer data
collection and maintenance, be assured that nothing, not process nor policy nor programs will stop a determined individual from circumventing established protocols to get the data in and the
overall job advanced to the next required step. In a smaller business this is even truer. Frequently the unwritten manifesto of the firm is “fight today’s fires ’cause they’ll be plenty more
tomorrow to worry about”. Fretting about something as strategic as data quality is a luxury most SMB’s would say they can’t afford.
The bottom line is this – in spite of all of the vendor promises or policy enforcement expectations ever spoken, constant customer data quality levels beyond roughly 98% is conceptually
possible but, practically speaking, realistically unattainable. As long as there is a human factor in the data flow this will be true. Accept it and you’ll have one less reason to grind your teeth
at night.
Does this mean then that SMBs, which are notoriously stretched much too thin on resources (human, capital, etc.) and time, should simply give up on improving customer data quality as an impossible
dream? Absolutely not – but they must temper their expected outcomes and set their sights with a strong does of reality. Understand that data quality levels are, in a sense, a highly
subjective matter. It seems obvious that getting a name and address correctly formatted and validated are key demands, but if your business is one which takes credit card orders on the internet and
ships to a client-supplied destination, customer name and address may not be your primary concern (because this data can be verified by the purchase approval or physical shipping process managed by
a third party firm). Such a business may need to focus on improving the inventory, order management or fulfillment data lifecycle instead. Just because we tend to think of names and addresses when
we hear the term “customer data” doesn’t mean that such information is a key driver for the business. Consider a privately held specialty machining firm which does $10-15 million annually
through a client base of 20-30 key customers. Managing names and addresses isn’t the biggest problem for this SMB – production management and engineering design that fulfill customer
requirements are, so, in effect, that is the “customer data” that this hypothetical company should get right.
It may also be that a company which does not have a large client list could have a significant vendor or partners list. Those may be the categories of data that need to be acted on to improve their
quality. (This example is one of the reasons that I prefer to use the term “client” instead of “customer”. Clients can be anyone or any business with which your business has some sort of
relationship. Customers tend to be only those individuals or businesses that have purchased something your company either produces or resells.)
SMBs can gain value and competitive advantage from data quality investments simply by knowing which data is mission critical to their firm. While it may be a stunningly obvious notion that customer
data should be of the highest possible quality, it isn’t always practical or wise to improve this data’s reliability to the exclusion of more significant categories of information.
So small business owners and senior managers should accept the inevitability of ongoing customer data management challenges – they will continue as surely as the sun will rise in the morning.
While doing so, however, they should closely examine their business and ask themselves this question: What is truly the most important data that their business generates? Whatever that is, focus
your scare resources on solving those data quality issues first and, very likely, the rest of the data will take care of itself.
