Published in TDAN.com July 2000
Corporations are demanding more and more functionality from all of their IT (information technology) systems, and meta data repositories are no exception to this rule. In this article I will
address the more complexed architectural challenges that arise with implementing a meta data repository that requires more advanced functionality. While most repository initiatives are not
attempting to implement these features, they are certainly the type of functionality that is being demanded from corporations. It is also important to note that these concepts can be implemented
separately or in-conjunction with one another.
Bi-Directional Meta Data
Typically the architecture of a meta data repository is one directional. Meta data sources (i.e. data modeling tool, extraction/transformation/load tools, front-end tools, etc.) flow into the
repository and are integrated together to satisfy the various needs of the business (business meta data) and IT (technical meta data) department. A bi-directional meta data architecture allows for
meta data to be changed in the repository and then be feed back from the repository into it’s original source. For example, a user could go through the repository and change the name of an
attribute (field) in one of the decision support system’s data marts. This change would then be feed back into the supporting data-modeling tool to update the physical model for that specific
data mart.
This architecture is highly desirable for two key reasons. First, it allows vendor tools to share meta data. This is especially desirable in the decision support marketplace. Most corporations that
have built a decision support system did so with “best-of-breed” tools, as oppose to integrated tool sets that are supplied by one vendor. However, these tools are not integrated with
one another and do not easily communicate. Even those tools that can be integrated traditionally require a good deal of resource intensive manual programming in order to share meta data. Second,
bi-directional meta data is attractive to corporations that want to implement a meta data repository on an enterprise-wide scale. By enterprise-wide meta data I mean companies that want their meta
data repository to store information on all of their IT systems (both decision support and operational systems). A bi-directional architecture would allow a corporation to make global changes in
the meta data repository and have those changes sweep throughout all of the tools of their enterprise.
As we can see a bi-directional meta data architecture provides a corporation tremendous value in allowing their disparate tools to communicate with one another. This need is why both the Meta Data
Coalition (MDC) with the Open Information Models (OIM) and the Object Management Group (OMG) with the Common Warehousing Metadata (CWM) have looked to resolve this situation by providing a industry
standard meta model. The meta model is the physical database model that will store the meta data. By creating an industry standard meta model third party vendor tools can plug their applications
into this model to share meta data with one another. Bi-directional meta data will become a reality as a global meta model standard emerges from either these groups. It is important to understand
that even once we have a meta model standard it will take time for the various software vendors modify their applications to work with this standard. This will enable these applications to have the
ability to share data, which creates tool interoperability. Each vendor’s tools have unique internal designs to them. As a result, even after we have a global meta model standard for decision
support, and the vendors modify their tools to support it, we will not have 100% meta data sharing. I do believe that it is possible to have a good 75% of the meta data be sharable, which is a much
better situation than what we have to deal with today.
Bi-Directional Architectural Challenges
There are three obvious challenges to implementing bi-directional meta data. First, is it forces the meta data repository to contain the latest version of the meta data source that it will feed
back into. Second, someone can be making a change to the meta data in the repository and at the same time someone else can be making a change to the same meta data at its source. These situations
must not only be systematically trapped but also resolved. Third, additional sets of program/process interfaces need to be built to tie the meta data repository back to the meta data source.
Closed Loop Meta Data
Corporations want to be able to integrate their customer relationship management system, decision support system, e-business solution to their operational systems to provide one integrated business
intelligence system. By integrating these systems together information, especially customer information can be shared throughout the organization. This enables corporations to provide new and
higher levels of customer service. In addition, service intensive industries (e.g. banking, financial services, retail) corporations want to have their systems make certain business decisions for
them, without manual intervention. For example, a retail company that sells consumer electronics would want an e-business system that would allow a customer to go through the internet to search for
and order whatever components they wanted. When the customer would select these components, a program interface would fire off to the customer relationship management system and other products that
the customer has previously ordered could be offered to the customer. Maybe if the customer hasn’t ordered the items in awhile a discount would be offered. Once the customer has finished
their shopping another interface would be sent to the corporate decision support system. This system would then update the customer’s credit rating and return this new rating to the
e-business system. It is this type of integrated, closed loop systems that are the mark of excellence.
Of course this desire for closed loop systems will directly impact our meta data repositories. A closed loop meta data architecture allows for the repository to feed its meta data back into a
corporation’s operational systems. This concept is similar to the bi-directional meta data architecture in that the meta data repository is feeding it’s information (meta data) into
other applications, or in this case operational systems. The closed loop meta data architecture is gaining more and more notoriety in corporations that want to implement a meta data repository on
an enterprise-wide level. This would allow a corporation to make global changes in the meta data repository and have it sweep throughout the operational systems of an enterprise.
Closed Loop Architectural Challenges
The closed loop meta data architecture adds some additional complexity to the meta data repository initiative. First, if the meta data that will be feed from the repository to the operational
system can also be maintained in the operational system this would require that the meta data repository contain the latest version of that operational system’s meta data. If not than the
user of the repository could not be sure of updating the latest copy of the meta data. Second, someone can be making a to the meta data in the repository and at the same time someone else can be
making a change to the operational system. These conflicts must not only be systematically trapped, but also resolved. Third, program/process interfaces need to be built to tie the meta data
repository back to operational systems. Currently few companies are attempting this architectural technique, however it is a natural progression in the architecture of meta data repositories.
