Meta data management and its use in enterprise data management have become critical information technology (IT) focuses for both global 2000 corporations and large government agencies. As these
entities look to reduce their IT portfolio and control their escalating IT costs they are turning to the technical functionality that a managed meta data environment (MME) can provide them. This
approach is very sound and the organizations that have built well-architected enterprise-wide MMEs have achieved a tremendous amount of success. Unfortunately, like most popular IT trends,
companies are making key mistakes in building and moving forward on their meta data management investments. One of the chief problems is that companies and agencies are not building an integrated
MME; rather, they are building lots of meta data repositories, none of which speak to each other and without an overall meta data management strategy. In this article I will discuss this
proliferation of unarchitected and disjointed meta data repositories and the problems that they cause.
The problem of disparate initiatives is not unique to meta data management. On the contrary, it is a common issue that has plagued many areas of IT. Technologies like data warehousing, enterprise
resource planning (ERP), supply chain management and all flavors of transactional systems have suffered with needless duplication and redundancy. The four most common problems created by disparate
meta data repositories are:
- Missing Meta Data Relationships
- Typically Built By Non-Meta Data Professionals
- Costly Implementation and Maintenance
- Poor Technology Selections
Missing Meta Data Relationships
In past publications I have discussed the different types of meta data (see Table 1: “Types of Meta Data”) that companies and government agencies need to properly manage[1] and the importance of having these meta data objects linked together. For example, it is very valuable for an IT
developer to have the capability to go to the meta data repository within the MME to look at the technical transformation rules (technical meta data) that are being applied to a particular physical
field name on a report that is being analyzed. Once the developer has reviewed this meta data they could then navigate through the repository to find the business rules defined by the business
users for that field. If a discrepancy between the transformation rules and the business rules exists then the developer could then use the meta data repository to contact the data steward that
defined the specific business rules and resolve this discrepancy. This is the true power of a meta data repository as it bridges the gap between our business and our IT systems, because in truth,
our business is comprised of our IT systems. When meta data is not managed from an enterprise perspective this type of click-through analysis is impossible as the relationships between the meta
data (both business and technical) are not being captured or maintained.
Poor Technology Selections
In my company we work with a good number of large corporations and vast government agencies. It is common that in the course of working with these groups to find many disparate meta data repository
or repository-like initiatives. For example, we have one client that has over 14 meta data repository initiatives (either in production or currently being developed) and another client that has
over 25 disjointed meta data repositories, most of which have significant monetary expenditures associated with them. Disparate meta data repository initiatives can come in many different flavors,
sizes and shapes. There are large repositories which utilize enterprise-level meta data integration tools or are even custom build (typically with Microsoft SQL Server or Oracle). Also, there are
lower technology meta data efforts that come in the form of Microsoft Excel spreadsheets, which, along with Microsoft Access are the most popular from of meta data repository technology. Does this
surprise you? If let’s consider this fact for a moment. Neither Cognos nor Business Objects are the most popular form of data warehouse access. Microsoft Excel is still the most popular data
warehouse access technology. This is one of the dirty secrets of data warehousing. With this fact in mind it is not surprising that Microsoft Excel and Access are being improperly used for storing
and retrieval of meta data.
Costly Implementation and Maintenance
Typically business units and technical groups need meta data to properly run their business and their IT systems. Therefore, these groups will not want to wait around for their company to start
building an enterprise-wide MME, assuming that they may not even have such a project in their current IT plans. As a result, they run off and build these disparate meta data repositories solutions
that are designed to only handle one or two specific problems/challenges. If executive management knew the cost of these meta data repository point solutions they would see that it far outweighs
the cost of a truly sound enterprise-wide meta data management repository. This situation closely follows the path that many companies have taken with data warehousing. For example, the average
company needlessly replicates their data warehousing efforts across many of their lines of business, as opposed to centralizing this function. It has been my experience that this approach typically
increases the costs of data warehousing by over 300%. These companies could provide all of the data warehousing functionality that they provide today at less than a third of the cost, if they just
managed their IT systems properly. The types of organizations that build point meta data solutions, are traditionally concerned by the cost of building an enterprise-wide meta data repository.
However, the cost of this enterprise-wide repository would pale in comparison to the costs of all of the disjointed meta data initiatives that are currently underway or in production (see Figure 1:
“Meta Data Management Costs”. Doing it right the first time is always less costly than doing it wrong and trying to fix it later.
Typically Building By Non-Meta Data Professionals
When government agencies and corporations have disparate meta data repositories, invariably most of these applications are being built by both consultants and employees that are not qualified to do
so. This is highlighted by the fact many consulting and software firms are entering the meta data management field even though they have little to no experience in this area. Meta data
professionals are like data warehouse practitioners. They are professionals that study their craft and have made it their full-time and in many instances life-work. You cannot take an operational
systems person or even a data warehousing person and expect them to work in the meta data arena if they are lacking the proper knowledge.
A meta data repository is not a data warehouse; rather it is an operational system. Building the meta data repository incorrectly will only lead to having to rebuild it again in the future. I have
published over a hundred articles in my career; however, only once did I ever write a predictions article where I attempted to say what will happen over the upcoming years. It was October of 1999
and I needed to write my January 2000 DM Review column. In approaching this column I thought that if I’m ever going to write a predictions article it must be for my first article of the new
millennium. How could I resist? My number one concern for this column was making sure that the predictions I made would come true, as I knew that someone would inform me if I missed any of my
“Nostradamus” forecasts. Therefore, my first prediction was that “During the 1990s, corporations raced to build their decision support systems as quickly as they could…in
their zeal to do this too many of these organizations neglected to build the architecture necessary to grow their systems over time.[2]”
We see today that data warehousing is becoming a standard practice in most companies and government agencies. Companies no longer ask if they should build a data warehouse, they ask how it should
be built. If you look at meta data repository and meta data management, it looks very similar to how data warehousing appeared in the early 1990s. The companies that used proper data warehousing
architecture and techniques have not be hampered by the high costs associated with disparate environments. Those companies that build sound enterprise-wide meta data repositories will have a
distinct advantage over those corporations and agencies that when down the disparate path.
[1] For more information on this topic see Chapter 2 in “Building and Managing the Meta Data
Repository”, David Marco, John Wiley & Sons.
[2] “Data Warehousing Trends for 2000”, David Marco, DM Review, January 2000.
