Published in TDAN.com July 2002
I thought that it would be valuable to take a look back and see where meta data management and repositories have come from and where the meta data management and repository industry are headed. As
the old saying goes, “We’ve come a long way baby!”
Many people believe that meta data and meta data repositories are new concepts, but their origins date back to the early 1970s, or in more general terms back to the first days of computing. When we
first started building computer systems, we realized that there was a “bunch of stuff” (knowledge) that was absolutely necessary for building, using, and maintaining information
technology (IT) systems. We learned very quickly that meta data existed throughout all of our organizations (see Figure 1). Meta data is stored in our systems, technical processes, business
processes, policies and people. Essentially, we knew that we had no place to put any of this information (meta data). At this point, we realized that we needed data about the data that we were
using in our computer systems.
Early Commercial Products
When the first commercial meta data repositories appeared in the mid-1970s, they were called “data dictionaries”. These data dictionaries were very “data” focused and less
“knowledge” focused. They provided a centralized repository of information about data, such as meaning, relationships, origin, domain, usage, and format. Their purpose was to assist
database administrators (DBAs) in planning, controlling, and evaluating the collection, storage, and use of data. One of the challenges that meta data repositories have today is differentiating
themselves from data dictionaries. While meta data repositories perform all of the functions of a data dictionary, their scope is far greater. The early meta data repositories (data dictionaries)
were mainly used for defining requirements, corporate data modeling, COBOL (common business-oriented language) and PL/1 (programming language one) data definition generation and database support
(see Figure 2).
Later a new phenomenon would enter the world of IT and forever change it…the personal computer (PC). When PCs burst onto the business scene, they changed the way companies worked and fueled
tremendous gains in productivity. CASE (computer aided software engineering) was one of the productivity gains. CASE tools were software applications that automate the process of designing
databases, applications, and software implementation. These design and construction tools stored data about the data (meta data) that they managed (see Figure 3).
It didn’t take long before the users of these CASE tools started asking their vendors to build interfaces to link the meta data from various CASE tools together. The CASE tool vendors were
reluctant to build these interfaces because they believed that their own tool’s repository could provide all of the necessary functionality, and, understandably, they didn’t want
companies to be able to easily migrate from their tool to a competitor’s tool. Nevertheless, some interfaces were built, either using vendor tools or dedicated interface tools.
In 1987, the need for CASE tool integration triggered the Electronic Industries Alliance (EIA) to begin working on a CASE data interchange format (CDIF), which attempted to tackle the problem by
defining meta models for specific CASE tool subject areas by means of an object oriented entity relationship modeling technique. In many ways, the CASE Data Interchange Format (CDIF) standards came
too late for the CASE tool industry.
During the 1980s, several companies, including IBM, announced mainframe-based meta data repository tools. These efforts were the first meta data initiatives, but their scope was limited to
technical meta data and almost completely ignored business meta data. Most of these early meta data repositories were just glamorized data dictionaries, intended, like the earlier data
dictionaries, for use by DBAs and data modelers. In addition, the companies that created these repositories did little to educate their clients in the use of these tools. Few companies saw much
value in these early repository applications.
In the 1990s, decision support emerged and soon convinced business managers of the value of a meta data repository, expanding the scope of the early repository efforts well beyond that of data
dictionaries.
The meta data repositories of the 1990’s featured a client-server paradigm as opposed to the traditional mainframe platform on which the old repositories operated. The mainframe vendors viewed
these new repositories as a threat since they greatly eased the task of migrating from a mainframe environment to the new and popular client-server architecture. The multiplicity of decision
support tools requiring access to meta data re-awakened the slumbering repository market. Vendors such as Rochade, RELTECH Group, and BrownStone Solutions were quick to jump into the fray with new
repository products. Many older, established computing companies recognized the market potential and attempted, sometimes successfully, to buy their way in by acquiring these pioneer repository
vendors. For example, Platinum Technologies purchased RELTECH, BrownStone, and LogicWorks, and was then swallowed by Computer Associates in 1999.
Where Are We Headed?
Meta Data Management Moving Mainstream
Currently meta data management and meta data repository development is at a very similar stage as data warehousing was in the early 1990s. In the early 1990’s, people like Bill Inmon were
essential in articulating the value of building data warehouses. At that time companies were beginning to listen and were starting to build their data warehouse investments. Meta data repositories
are moving in much the same direction today. In fact at Enterprise Warehousing Solutions (EWS) we are doing more meta data repository development now than at any other point in our company’s
history. Companies are beginning to realize that they need to make significant investments in their repositories in order for their systems to provide value.
Meta Data Repositories Providing Knowledge Management
All corporations are becoming more intelligent. Businesses realize that to attain a competitive advantage they need their IT systems to manage more than just their data; they must manage their
knowledge (meta data). As a corporation’s IT systems mature, they progress from collecting and managing data to collecting and managing knowledge. Knowledge is a company’s most valuable
asset and a meta data repository is the key to managing a company’s corporate knowledge (for more information on this topic see “A Meta Data Repository Is The Key To Knowledge
Management”, David Marco, DM Review, December 2000).
Maturing Meta Data Integration Products
There has been no tougher critic of the meta data integration vendors than myself and I still believe that these vendors are neglecting their most important user: the business user. With that being
said, in the past year I also have seen across-the-board improvements by almost all of the vendors in this area. New vendors like Data Advantage Group are coming onto the meta data integration
scene with new and exciting products. In addition, the more traditional meta data repository vendors like Computer Associates and Allen Systems Group have all dramatically improved their product
lines.[1]
Meta Data Repositories/Management Continue Moving Mainstream
To illustrate how far meta data repository development has come, around 9 months ago I was asked to speak for one day to a group of approximately 15 IT senior vice-presidents of banks. Their number
one technology issue was meta data! When I spoke on meta data many years ago, we were lucky to have 15 IT developers in a talk. In most every Fortune 500 company there exists massive amounts of
redundant data (I have experienced that the average company has 4 fold needless data redundancy), needlessly redundant systems, and tremendous data quality problems. Fortunately, executive
management are starting to realize that these problems result in a tremendous cost drain for their companies. These same people are looking to control the costs of their IT departments through the
use of meta data repositories. As a result, meta data repositories and meta data management are continuing to move up corporations’ IT priority lists.
[1]If you are assessing these vendor’s products you may be interested in a third-party evaluation. Information on EWS’ 150-page
comparative study of these products can be found on the EWS website at www.EWSolutions.com or by emailing Info@EWSolutions.com.
