For multiple reasons many IT organizations have “down-sized” over the last few years. The result is that we are left with less development resources to support the building of new internal
applications. While business needs have increased, the building of an internal application such as a Meta-data repository is less of a reality than it has been in the past, making the option of
“buying” a commercial Meta-data repository more attractive to many CIOs.
In the past two years I have been asked by two large international companies to leverage my experience to help them purchase of a commercial meta-data repository tool. As you would expect both
companies had extensive meta-data, stored in tens of tools and in the heads of many thousands of business and technical staff members (carbon based life forms). While meta-data was stored in
thousands of places, like peoples heads, very little was accessible, sharable or re-usable across the enterprise. Bottom line was that the meta-data and associated knowledge was not reusable
between organizations, projects, or people consistently which equates to lost opportunity time and a larger TCO of business operations.
Both firms had tried multiple times to solve the meta-data acquisition and delivery problem, including building their own centralized meta-data repository applications. It seems that building their
own meta-data repository was not successful over time for reasons such as:
- Funding was not sustainable (ROI was not defined or value documented),
- Applications built internally were not scalable or easily extensible (great for one team not for many),
- Did not include enough technology sources of meta-data (sourced from only one database),
- Did not include business meta-data (sourced data-base and ETL logic only),
- Did not include enterprise or logical data or process models,
- Did not create a user interface that was effective for either technology or business personnel,
- Required significant human resources to manage and maintain (operations and administration headaches)
It seems that both of those organizations had been successful in building meta-data repositories that were point solutions for one or two development teams. Yet, both had relatively poor
functionality to view the meta-data. As my good friend Bonnie O’Neil would say, those implementations remind one of the equivalent of a roach motel …”you can check in but you can’t check
out”. In fact, one of those companies stopped the implementation of the internal project when they estimated that it would take the QA team 24 person weeks just to “test” the acquisition code
for a new source.
Yet, the “killer meta-data application” is based upon the acquisition of the meta-data, but one the effective search and delivery or presentation of the meta-data. Meta-data should enable the
information age worker; both business and technical staff members, to be more effective and productive.
Now, the two corporations I mentioned are very large, highly distributed, and complex environments. Your environment and meta-data needs are likely less complicated. Just remember that the
meta-data repository program, like all programs, require good people, integrated processes, and then effective tools. A meta-data repository application is just a tool.
Many companies have been very successfully by developing their own meta-data repository applications. In fact, most of the finalists in recent years DAMA International Meta Data awards had
developed their solutions internally. However, with the staff downsizing that has occurred over the last few years, many are now considering to purchase a commercial meta-data repository product.
The good news is that there are many Meta-data Repository applications on the market today. Some are relatively new and some have been around for 10 to 20 years. If you are considering a commercial
meta-data repository I suggest you start your research with the Gartner Magic Quadrant for Meta-data Repositories, 2004. That report lists thirteen products and there are likely a few others that
you may want to consider as well.
Meta-data Repository Acquisition Approach
If you are considering the purchase of a commercial meta-data repository, the following is an approach that has lead to success in both the cases mentioned above.
You may find that a few of these steps can be consolidated within your environment. While I have been requested to consolidate steps, I would never recommend eliminating the Preparation or
Requirements steps. It seems that many organizations fail to sustain their meta-data programs by only focusing on the technology tools. The trap is to just go get a software product and life will
be good. At the risk of repeating my self, the critical pillars for a successful meta-data program are:
- Organizational (for alignment and staffing commitment)
- Process (effective workflow capture and standards)
- Governance (monitoring of the processes and communications)
- Tools (lastly, effective tools to enable a productive environment)
The meta-data products on the market have significant differences in source acquisition, functionality, and reporting delivery. For example, all of the products can acquire meta-data from a
database, but not necessarily from the database or version you have installed. Suppose your organization uses 3 different DBMS, which one is most important to start with? All of the commercial
products have functionality to import physical level meta-data. But suppose your most critical requirement is to import and manage business and enterprise model level meta-data. Just a few of the
commercial products can be extended to implement business level meta-data functionality. Thus, you must have a definitive list of functionality requirements in order to define your meta-data
evaluation criteria and weighting.
Preparation Step
This step is critical to define the organizational and governance pillars. The objective of this phase is to gain agreement on the goals, staffing, and communications the Meta-data Program. The
activities of the Preparation phase include:
- Establish the Meta-data Program Vision and Guiding Principles
- Define your meta-data value proposition and ROI
- Establish the Meta-data Acquisition Project Charter
- Define the participating Organization, Roles, and Responsibilities
- Develop the Meta-data Acquisition project plan
- Assess status of the Meta-data Governance program
Requirements Definition
The objective of this phase is to gain agreement on the requirements of the Meta-data Acquisition project and establish the vendors that will be investigated. The activities of the Requirements
phase include:
- Define the existing and future workflow and processes involving the definition, integration, and presentation of meta-data
- Define the detailed business/technical requirements
- Define the Meta-data Governance Process Flow
- Define the Meta-data System-of-Record and toolsets
- Develop a straw Meta-data Meta Model
- Review Meta-data Market/Select Vendors
- Develop RFI and timeline
Publish Request for Information (RFI)
The objective of this phase is to select the final short list of vendors that will be solicited for an RFP. Many projects may eliminate this step and move directly to the next phase. The activities
of the Publish RFI phase include:
- Publish RFI and Manage Vendor Q&A
- Establish RFI evaluation criteria
- Manage Vendor RFI responses
- Evaluate RFI responses
- Select vendors for the RFP step
Publish Request for Proposal (RFP)
The objective of this phase is to publish an RFP and receive back complete responses that can be compared and evaluated. The activities of the Publish RFP phase include:
- Develop RFP package
- Publish RFP and Manage Vendor Q&A
- Develop RFP evaluation criteria and weighting
- Manage RFP Responses
- Schedule and conduct Vendor demonstrations
RFP Evaluation and Recommendation
The objective of this phase is to evaluate the RFP responses, and select vendors for the POC. The activities of the RFP Evaluation phase include:
- Conduct internal response reviews
- Conduct Vendor demonstrations and response review
- Complete individual evaluations
- Complete team evaluation and recommendations
- Complete reference checks
- Finalize pricing scenarios and options
- Begin Contract negotiations
- Schedule Proof-of-Concept, if necessary
Proof-Of-Concept (POC)
The objective of this phase is to confirm the Vendor functionality and recommend a single Vendor solution. This phase may not be necessary if the evaluation team is confident that a single Vendor
has the most complete solution to the requirements. However, I have seen where a team could not make a clear decision. Many vendors have significant strengths in different functional areas that can
produce a near equal evaluation scoring. If that is the case, then a short POC is needed to finalize a recommendation. The activities of this phase may include:
- Develop the use cases and source meta-data for the POC
- Define and install hardware and connectivity to sources
- Schedule POC facilities, Vendor, and internal support staff
- Develop POC evaluation criteria and weighting
- Conduct POC
- Evaluate POC and score Vendors
- Final Vendor Recommendation document and management presentation
Install and Pilot
The objective of this phase is to install the meta-data repository product and validate the functionality of the product against the anticipated staffing, processes, and governance (tools against
the other 3 pillars). This is a critical step to ensure that we have alignment in the people, processes, governance, and tools. While all steps are important, this is the most critical step and
must be planned effectively. The activities of this phase may include:
- Production site installation and connectivity software installs
- Project planning and staffing
- Develop use cases for the Pilot
- Product training
- Engage the staffing, Governance, and best practices
- Conduct the Pilot use cases
- Monitor execution of the use cases
- Document use case successes and challenges
- Document the success, weakness, and enhancements in
-
- Organization and Staffing
- Processes and standards
- Governance
- Meta-data tools
- Recommend changes to the product infrastructure, usage, and presentation
Meta-data Governance
Meta-data Governance is not a one-time step like the others described above. Governance includes staffing, development and business processes, workflow, standards, and best practices. Governance
starts early in the program and never ends. The activities that may be included are:
- Managing the meta-data ownership or stewardship program
- Defining the meta-data repository application architecture and life-cycle
- Defining and managing the meta-data quality metrics and processes
- Defining the meta-data standards, security, and best practices for acquisition, management, delivery, and change management
- Managing the meta-data systems of record, source tools, interfaces, workflow, and operations
- Define the meta-data configuration management practices
- On-going communications of the meta-data repository program
- Integrating the meta-data repository into the business and technical workplace
It may seem like there is a lot that needs to be done, and there is. Implementing a meta-data repository is a complex effort. However, you can divide the problem into smaller steps as I have
outlined above. I have worked with organizations that have attempted to “boil the ocean” and spend years with no results to show after lots of hard work. From experience I can say that the above
effort can be accomplished in 3 to 6 months, depending upon the scope of the organization and resource constraints.
