IRM Test Number 2 – What Can We Do to Change

Published in TDAN.com July 2003

(Re) Introduction

More than a year ago, I wrote an article for my publication called The IRM Test: How Did “IT” Get This Way? A Self-Help Test. The article was written quickly, with
tongue-in-cheek, and I was curious as to what the response would be. Response was great.

This article is written in the same spirit as the first IRM Test. Truth be told … the past year-plus has been enough time for all data professionals to read that article, take that test,
evaluate their position and make drastic improvements so they can score better this time around. Anyone that read the article and took steps to improve their score can stop reading this article at
this point.

My guess is that I lost … nobody.

The “IRM Test Number 2” is a new series of simple and potentially fun (?) questions that you can ask yourself about your company to give you a clearer understanding of why you are having problems
managing information assets. Just to remind you, … this test is not meant to be taken seriously, unless of course, you decide to ask some of these questions to the powers that be in the IT
area of your company. It may be interesting to compare their answers to yours.

The Point Scale

Let me remind you of the point scale I used in the previous IRM Test. On my IRM Tests, companies get more positive point credits for positive activities than they have debited when they are not
taking steps to improve how they manage their information resources. One would think that should lead to higher overall scores, which as you will find out in the end, don’t have any meaning
at all. Determine your test score by applying the following point scale. Do your best to select only one answer for each question.

-1 point for each “A” answer. One point is taken away because your company recognizes that it has a problem managing its information assets and decides to take
limited action or not do anything about it. This score is very frustrating for experienced information resource and data managers for a couple of reasons. The obvious reason is that if you selected
this answer, it is probably very difficult ­ if not impossible ­ to practice what you preach. The second reason is that you must be concerned that your job or the practice of information
resource management at your company could be in jeopardy.

0 points for each “B” answer. No points are taken away but no points are added. Your company has applied a temporary or partial fix to an information management
problem. This is better than doing nothing at all but may indicate a lack of knowledge about how to develop or implement best practices. Things aren’t getting any worse, but they aren’t getting
any better either.

+3 points for each “C” answer. Three points are added because your company follows best practices when it comes to managing your information assets. In this
environment, the information resource manager has a real chance to be successful. Look at how you answered this question and try to apply the same philosophy when solving other problems identified
through the questions of this test.

Now for the test …

The Business Intelligence Score

Do we (as a company) have standard procedures and methods for how we identify/record, approve and finalize data requirements for our data warehouses and data marts?

1. A few times, we started creating standard procedures for warehouse development across the company. The effort we put into standards typically comes during one of our pushes for improved quality
   corporate wide. Truth is … when push comes to shove and the business units need their data warehouses and mart, they always pick their own tools, follow their own standards and typically end
   out creating a whole new set of data.
2. We have standard procedures that are more like guidelines for collecting data requirements and creating data warehouses and data marts. The company has standardized on several tools like data
   modeling, reporting and ETL. We share data when it is easy and doesn’t require too much clean up work. We will be the first to admit that we still do not do a very good job of sharing data or
   knowledge about data..
3. Our company follows a set of internally developed procedures for the requirements gathering for all data warehouses and data warehousing efforts. These procedures include the use of company
   standard tools. Since these standards were created and blessed by Senior Management, warehousing efforts are required to follow the procedures or they risk the loss of funding.

How do we identify the right people to involve in discussions about what data we are using, what their data means, how the data will or can be used in our data warehouses?

1. Most folks in our company have been around a long while and we know, from our years of working together, who the right people are to involve in our data warehousing efforts.
2. We publicize our efforts and hold large meetings early in our warehouse/mart development processes to bring anybody in the business and IT community who has an interest in the project together
   to discuss how the data warehousing effort is going to proceed, how it may impact them, and what benefits they can gain.
3. Each of the data sources, types of data and business community users are represented by at least one steward who is accountable for making certain that the data that they steward is managed and
   used properly. This steward information is recorded in a database that we use just for this purpose. That is how we know whom to involve in the data warehousing process.

What do we do with data quality issues that are discovered in the analysis and movement of data from their original source into our warehousing environment?

1. We clean up the data as much as we can as it is sourced into our data warehouse. We adjust and sometimes correct data that is missing so that it passes our tests to make it into the warehouse.
2. When we move data into the data warehouse, we test the data at the source first. If the data is valid it gets moved into the data warehouse. If the data is not valid, we do our best to correct
   the data at the source and we report to our warehouse users what percentage of the data was not corrected, corrected and what business rules were applied to correct the data.
3. Our company analyzes the quality of each data source in the very early phases of data warehousing / data mart efforts and we use this analysis when identifying what data can and cannot be used.
   Data deficiencies and corrective actions are noted throughout the business intelligence lifecycle. We have processes in place to load only “high confidence” data into our data
   warehouses and we always aim to correct defective data at the source.

How confident are the data warehouse users and knowledge workers that the data in their warehouses or marts will accurately answer the questions that it supposed to be able to
answer?

1. The “power users” say they understand the data well enough to do their jobs. Keeping those power users happy is one of our most important BI objectives. The power users are the
   biggest users of the data and they are closest to the real data issues so they can usually work their way around problems. The rest of the data warehouse user base uses canned or parameter driven
   reports and we assume they trust the data because they are getting and supposedly using the reports.
2. The warehouse users are pretty confident that the data is good. We constantly poll the “power users” and the casual users for data issues that are causing them problems and
   suggestions on what data they need and how the data should be collected and made available. This feedback is good and we address as many of these issues as we can.
3. There are formal processes in place for elevating the use of the data in our warehousing environments. Regular meetings are held with the “power users” and casual users to get their
   input on the data already in the data warehouse and data that is being added or extended to existing warehouses. The formal processes get average business user active in data issues and has been
   shown to improve their reported comfortable level with the data in the warehouses. The users know that we are building data to serve their purposes.

The Meta-Data Score

Does our Senior Management really understand what meta-data is and the impact meta-data has on all of our business-focused information technology projects?

1. Senior management say that they know what meta-data is and that they know it is important for “data warehouses” and other IT projects, but they have not allocated any of our time to
   improving our use of meta-data or given us any time or extra resources to focus on building an appropriate meta-data program.
2. Senior Management has been educated in the many ways that we can use meta-data to improve the quality, value and use of corporate data. Senior Management repeatedly asks the two key questions
   – “How much will it cost?” – and – “What is the ROI?” of a meta-data program. We, as a company, have not “sized up” a meta-data program so we
   have not yet tried to estimate the cost. We also have been actively searching for information about meta-data ROI. That info, in a format that would be acceptable to our Senior Managers, has been
   hard to find and always seem to use “fuzzy math” to estimate their numbers.
3. Our company’s Senior Management understands the need for meta-data and has given the responsibility of managing meta-data to our data administration and data management team. We do not
   have a large budget allocated to developing a meta-data program but we are taking active steps to develop our meta-data resources and distribute meta-data into the hands of the business and
   technical staff that need it to perform their jobs.

Do we know what meta-data we have, where the meta-data is located, the quantity and quality of that meta-data, and how we can leverage the meta-data to understand the data and to
increase revenues and decrease costs?

1. We know we have meta-data all over the place. As a company we use at least two data modeling tools, we have three different ways to move data, we use at least three different query tools and, I
   think, we are (or were) part of the database-of-the-year club with IDMS, IMS, DB2, Oracle, SQL Server … who did I leave out(?). We have not tried very hard to use the meta-data in these
   tools … and therefore we do not focus our attention on the amount of meta-data in the tools or quality of the meta-data that is entered into the tools.
2. As a company, we know what meta-data is most important to us and we have been focusing on “standard” tools partly to elevate our ability to use the most important meta-data. We
   spend a lot of time putting sound business definitions into our modeling tools and in documenting how we are using our data transformation tools. We also have an active list of validated reports
   and queries from our BI environment but keeping this list up to date is a difficult. We think we have a good start on managing the meta-data but we need some help getting the meta-data in the tools
   into the hands of the data users.
3. We have evaluated our meta-data needs and have determined our objectives for our meta-data program. We have taken a look at each of the tools we use and we have made the ability to manage and
   access the meta-data in the tools a major contributing factor in whether or not we use the tools. We have created standards for how meta-data is entered into the tools and we have identified a way
   to get the meta-data into the hands of the people that need it.

Have we defined specific resources that focus either all or a large percentage of their time implementing a meta-data program, managing meta-data and helping us leverage the
benefits?

1. Our company has grown smaller over the past few years and individuals are being asked to carry more responsibility in the same period of time. Keeping the business running and keeping our main
   applications running has been a top priority – Meta-data has not been a top priority – So I would have to say “No”, there aren’t really any specific individuals that
   have been given the responsibility to make a meta-data program happen.
2. Meta-data management has been added to the list of responsibilities of our data management and data architecture staff. We are in the process of putting together a plan to manage meta-data but
   thus far not much has been acted on. The meta-data effort is initially starting in the IT area with a focus on helping IT to reduce data redundancy and improve our technical processes in the
   development of our data warehouse environment. Business folks are not asking for meta-data or funding the efforts. We figure if we start small by managing meta-data for IT, eventually the business
   users will recognize the need for meta-data and jump on the bandwagon.
3. We have a group of individuals that specifically focus on meta-data management as well as data quality, data modeling, data movement, … all aspects of the data lifecycle. These
   individuals have the responsibility of identifying the “right” meta-data to manage, building standards for the collection, recording, and distribution of meta-data, and they are also
   responsible for making known throughout the organization what meta-data is available, the importance of meta-data, and how meta-data will help staff to leverage the company’s data.

Do we carefully analyze (before buying) new tools that are being brought into our IT environment based on the information they record (meta-data), the openness of their information
models (meta-models), the ability to access the information that is recorded in the tool and it’s ability to integrate with tools already in the environment?

1. Our company’s IT shop has gone from centralized to de-centralized and back a couple of times. Each of the business areas have the ability to choose tools that they need to complete their
   projects. Only recently have we looked at standardizing on specific tool sets. Typically meta-data is not a top factor in the selection of tools in our environment. We, the staff that cares about
   meta-data management, might get a chance to voice our opinions but that doesn’t necessarily mean that we carry any weight in the decision making process.
2. We, the “data folks” (as we are known), have a large say in the tools we select to be used by other “data folks”. We always look at how well the information entered into
   the tools can be leveraged by individuals that don’t necessarily use the tool (data movement info for non-ETL users, data definitions for non-data modeling tool users, …), what
   meta-data goes into the tools, what meta-data we can get out of the tools, how meta-data may move from one tool to another tool.
3. Before a new IT tool is brought into our company, the group bringing in the tool must get permission from a Technology Committee. This committee meets monthly and follows a strict set of
   guidelines that include analysis of existing tools in our environment (to make certain that we leverage existing licenses or take advantage of license discounts), analysis of the ability to educate
   users on what information needs to be entered in the tool and how to enter that information, and the group has now started to pay particularly close attention to how the meta-data is stored in the
   tool, how that meta-data is managed and its interoperability with other tools in our IT shop.

The Data Stewardship Score

Does our company have an information asset operating policy that has been approved and is being enforced by Senior Management and made a part of everyday business and information
technology operating procedures?

1. I am not certain that we have an information asset operating policy per se, yet I am pretty certain that, with recent legislation changes, we have developed data security policies and data
   privacy policies. If we have an information operating policy, there is no evidence that it is known or is being followed by most of our data professionals or business managers.
2. We have an information policy that was written several years ago, it was blessed by Executive Management and added to other operating policies. These policies have been updated several times in
   the past couple of years and they represent how the Executive Management states that the corporation should act. The problem is that the policy has not been communicated and shared with all of the
   employees of the company and there is presently no formal method or means of enforcing the policy.
3. We have an information asset operation policy that is an active part of our information systems and information management environment. Everybody in the company (including all new hires) has
   been given a copy of the information policy or has been educated on how to access to the policy through our intranet. Every year employees are evaluated partially on how they abide by operating
   policy and they are required to review all operating policies (including the information asset policy) so that they understand how strongly Executive Management feels about the importance of
   enforcing corporate policy.

Does Senior Management recognize that managing information assets requires making individuals formally accountable for the management of these assets in the same way that others in the
organization are held accountable for the management of human assets, financial assets, knowledge assets, property assets, … ?

1. Because of issues raised by auditors and other regulatory parties regarding our use of data, we are certain that Senior Management is well aware that data assets require formal accountability
   just like the other assets. However, to respond to audit and regulatory issues we often find ourselves solving the problems specifically to appease the auditor and get-by on legislation and
   regulatory concerns. We never do more than we need to do to get by.
2. Our company and Senior Management have made several attempts at starting data stewardship programs to begin to apply formal accountability to the management of data resources. Although data
   stewardship has not been adopted across the company there are pockets of individuals that meet from time to time to discuss and resolve business definition of data and data quality issues.
3. Senior Management recognizes the need to apply formal accountability for the management of data resources and has done something about it. Management has supported the development of a data
   stewardship program that is being adopted across a large portion of the organization. The program consists of formal definition of steward roles, responsibilities and accountabilities. The
   information about who is accountable for the definition, creation and usage of data is recorded in a corporate database to be accessed by whoever needs it to do their job.

Have we defined data steward roles and responsibilities as they relate to 1) who and how data is defined for business use, 2) who/how data is created and managed throughout the
information lifecycle, and 3) who and how data is used by the knowledge workers of the company?

1. People at our company know what their roles and responsibilities are when it comes to managing data. For each system and database, someone is responsible for defining the data for that project
   and making certain that the data is used properly. There is no dire need or plans to formalize this process. When data-oriented conflicts occur, the person with the most clout typically wins out
   … either that or they create new data that serves their purpose.
2. We have defined data steward roles and responsibilities for specific areas of the company, projects and data warehousing activities. The roles and responsibilities have been tweaked for each
   group and there is not presently a consistent way of applying stewardship across groups, projects or across the company. Decentralized data stewardship seems to be working at least for some of the
   areas and projects. At some point we may consider pulling together our programs into an enterprise effort.
3. Our data stewardship program is in effect across the company. The roles and responsibilities have been defined once and are being used by each of the business and technical divisions and
   departments. Steward roles have been approved for the definers of business data, the creators and maintainers of data, and the use of data. Meta-data, data that relates staff or their positions to
   their specific responsibilities, is recorded in an enterprise resource, our Steward Management System.

Have we identified when stewards should be involved in data-focused activities and has involving data stewards become part of our information technology team’s structured
development lifecycle?

1. Our company believes that everybody is a data steward, that is, everybody has a level of responsibility toward the management of data resources. Therefore, stewards are involved in each step of
   our application development lifecycle. Since everybody is responsible, there has been no reason to tag specific parts of our methodology and lifecycle with specific steward involvement.
2. We follow a structured development lifecycle and we have defined when the stewards should be involved every step of the way. Sometimes the level of defined involvement is enforced, sometimes it
   isn’t. It really depends on the situation at hand and the impact involving the stewards have on our ability to deliver within the given time schedules for our projects.
3. Our company follows a strict set of lifecycle and methodology tasks for all new projects and database design efforts. The stewards that have been identified as definers, creators, and users are
   involved in each of the appropriate steps for their level of accountability and the stewards are required to sign off on how the data is being managed each step of the way. This slowed us down at
   first but we have become accustomed to involving the “right” people at the “right” time. … And you know what? Data stewardship has had a direct impact on our ability
   to manage data redundancy and improve the quality of our data.

Making Sense of Your Scores

As you may recall from the previous IRM Test (and as I mentioned earlier), this test differs from other “self-help” tests you may have seen in that there is no total score. Each section and
question must be taken at its face value.

If most of your answers scored a “-1”, consider taking the next train out of town unless there are indications of major changes on the horizon. Remember, again, that
these changes do not come quickly and often they are treated with “lip service” for a long while prior to any action being taken. The hurdles that must be jumped to solve problems identified in
these questions are hurdles of value and attitude. If you can provide the impetus to change these attitudes, you may become a true hero. Many have lost their jobs or had their jobs eliminated
trying to be this hero.

If most of your answers scored a “0”, there is a good chance that your company can turn the corner and improve at managing your information resources. These answers
indicate a beginning and a desire to change for the better and indicate that there are steps being taken to do so.

If most of your answers scored a “+3”, good for you and good for your company. To answer these questions this way means that your company recognizes the need to manage
its information resources. In this situation, the information resource manager has a chance to excel and exceed expectations, new ideas can be introduced (and taken seriously), and job security is
not a concern.

I hope that this second test was helpful and entertaining and that you didn’t take it too seriously. As with other “self-help” tests, this was only written to make you think, or think twice,
about what you (and your company) are doing.

Managing information resources through the structured disciplines of business intelligence, meta-data management, and data stewardship can be an important job in your company only if those
individuals who are making the decisions view it as an important job. Over the years, as experienced data managers, information managers and knowledge managers move up in the ranks at their
companies and companies become more dependent on their information resources to stay ahead of the competition, we will continue to see companies making great strides in their ability to manage
their data and information resources.