The time wasted spent collecting and validating data can approach 80% of the total effort for managing data. This estimate was shared by several speakers at the DGIQ 2022 conference in San Diego. We had previously estimated this “waste tax” to be in the 20-30% range. No matter what the actual numbers bear out, this waste is a huge problem and it’s getting worse. At DGIQ, there was also considerable discussion on the bureaucratic burden modern Data Governance has brought into corporations. Another source of waste. Other than references to “Data Governance 3.0” as the next new thing, we did not hear much in the way of practical solutions.
This brings us to Lean Governance as a way forward. MetaGovernance has dedicated this column to the Art of Lean Governance. Disclaimer: lean is not a synonym for agile. In this writing, we are going to revisit the factory walkthrough as a process to detect and eliminate waste.
Lean Thinking began in automotive manufacturing. Lean Manufacturing and the emerging Theory of Constraints revolutionized this industry. The hallmark of this approach was to build quality into the products with the least amount of time and materials. The goal was zero defects. As this approach gained acceptance, company after company began the journey to lean with simple walkthroughs of their manufacturing floor looking for waste. Among the first problems to be spotted were the growing piles of work-in-process inventory next a machining workstation. Somebody upstream was working faster than the process could handle. That explained the bottlenecks and production delays. Factories that went lean ran smarter about how they managed existing resources—equipment, inventory, and people. Less truly was more. It still is.
What does this have to do with Data Governance? Everything. We can learn much from the factory movement to lean by cutting procedures—including bureaucracy—that contribute zero value to the customer. Back then, quality metrics were redefined. Inventory management was king. Factories returned to the basics of manufacturing a quality product at the least cost. All it took was a massive paradigm shift.
We see Enterprise Information Management (the modern data factory) as the total set of processes and technologies used in. This includes all the procedures, controls, and safeguards built in to reduce risk and waste. With their stated goals of data quality and regulatory compliance, many data factories today are failing due to bureaucratic burdens.
Below, we show where our data factory walkthroughs uncovered waste in two key areas: manual data reconciliations and incomplete awareness. These examples were comprised of interviews and general observations from a broad group of stakeholders.
Manual Data Reconciliations
Show us a spreadsheet used for data collection or validation and we will show you waste. Given the amount of time and money companies spend on data quality tools, it is still mind-boggling how often we observe manual data reconciliation. This process usually involves multi-tab spreadsheets that are designed to validate data used for reporting or operations. They still exist because the data consumers do not trust the data, or their control procedures mandate evidence. Why aren’t these reconciliations automated, and why aren’t the results shared? At one client we counted the exact same reconciliation happening across 6 departments on a daily basis. In this case, the added cost was estimated at over 200 hours total per month. The solution was to leverage existing technology for the reconciliation and publish and share the results.
Incomplete Governance Awareness
People are trying to do their jobs without sufficient detail on the owners, consumers, system of record, definitions, and copies of data. We call it ‘Awareness.’ The massive projects to build glossaries and lineage can become outdated due to reorganizations or system changes. Beside the sheer amount of wasted effort spent chasing down answers, the bigger cost is exposure to risk.
This was highlighted recently by a treasurer who was horrified to learn that a market risk department had been using the wrong data source for model validation for over two years. While the treasurer knew that market data vendor could not be trusted for this class of instruments, not all of the data consumers had this knowledge. Everyone was made aware of the problem following an adverse regulatory finding. This damaged the reputation of the organization. Higher audit fees and tighter scrutiny added insult to injury. All because the true system of record was not known.
The Solution: Know Your Inventory
MetaGovernance utilizes an Awareness Matrix as part of our Lean Governance methodology. Using the factory analogy, the key to success is knowing your inventory. Rather than parts and widgets, your inventory is data. Both factories require inventory classification systems. When working with dangerous materials, factories require safety labels and handing instructions. In the data factory, confidential and PII data require the same. In data and information, this last point is a hallmark to Information Governance.
In the world of data, the sheer volume of moving parts requires a precise classification system. In many clients we see upwards of 20,000 individually named columns, many of them synonyms due to spreadsheets. It is virtually impossible to maintain awareness at this level. Instead, we group data into like subject areas, or domains. Depending on client industry and desired level of precision, we typically see under 100 subject areas. We then establish governance awareness at the subject area level. This awareness is tracked as metadata and is comprised of owners, consumers, security classifications, retention, system of record and risk levels. For critical or sensitive data used in controls or of increased vulnerability, we track lineage of the sheer number of known copies across structured and unstructured data.
Invitation to Participate in Lean Governance Project
This level of awareness and control makes a measurable difference in delivering value to the data customer while reducing risk and cost. MetaGovernance is looking for joint research projects for a book we are writing on the Art of Lean Governance. We welcome the opportunity to explore (confidentially) ways to eliminate possible waste and risk in your data factory at no cost. Please contact us if interested at metagovernance.com.
