Data Rich, Information Poor? Focus On Right Metrics

On a recent flight to Washington DC, I was seated next to an executive who was looking ever so zealously into her laptop screen at some kind of executive dashboard with nice looking charts. Piqued
by my interest, I grabbed the earliest opportunity to introduce myself and how Business Intelligence Solutions were my bread and butter. She was delighted at the background and said that she was
“looking at some KPIs from a recent eMarketing campaign, an email blast to some leads and customers”. She talked about how overwhelmed she was with the different sets of numbers and sought a way
to ferret out patterns that’d help her business unit make better decisions. Later, recollecting on our conversations, I realized that one hears so often about the importance of metrics, key
performance indicators (KPIs), measurement, etc. that it almost sounds clichéd to use the terminology. But businesses cannot function without some kind of indicators of current performance
and future forecasts of performances. I still remember when I first set foot on the Six Sigma road, someone had summed the business philosophy as:

• If you can’t measure something, you really don’t know much about it.
• If you don’t know much about it, you can’t control it.
• If you can’t control it, you are at the mercy of chance.

This sums up the importance of measurement, and how measured data translates into information, which finally morphs into knowledge. But before we
explore why we need to measure and what we need to measure, it’s good to understand the different nuances of measurement systems:

• A Measure is a quantitative indication of the extent, amount, dimension, capacity or size of some attribute of a product or a process. It is a single data point (e.g.
  number of defects from a single product review).
• Measurement is the act of determining a measure.
• A Metric is a measure of the degree to which a system or process possesses a certain attribute. Metrics relate data points to each other (e.g. average number of
  defects found in reviews).
• • Metrics can be directly observable quantities (the number of source lines of code, number of man-hours, etc.) or can be derived from one or more directly observable
    quantities (defects per thousand lines of code, etc.).
• An Indicator is a metric or series of metrics that provide insight into a process, project or product. So indicators are metrics in a form suitable for assessing
  project behavior or process improvement. An indicator may be the behavior of a metric over time. For e.g. number of trouble reports written and resolved over time, number of requirements changes
  over time, etc. Indicators are used in conjunction with one another to provide a more complete picture of project behavior.

One can’t quantify how well the business processes are working without some measure of the current performance and a baseline to compare against. Metrics help us better control our projects and
learn more about the way the organization works. They enable targets to be set, success to be assessed, ROI to be tracked, ongoing viability to be ascertained, and lessons to be learnt. But the
goal is to have relevant metrics for a certain business unit depending on level of the decision making involved (operational, tactical or strategic).

Best Practices for developing your business metrics

1. Get the business leaders/customers involved

Since the strategic vision of the company flows top-down, the leaders need to be involved in how the metrics are linked to achieving their goals. The executive leadership define the what,
why and eventually how of the business and the key success factors metrics will directly feed into the “how” of the business and should have answers to the following:

• What should you measure?
• How often should you measure?
• Who is accountable for the metric?
• What should you use as a benchmark?
• Do the metrics reflect strategic business drivers? Do they correlate to competitive advantage?

Also it is important to talk to the customers because at times too many performance metrics look inward, reflecting what operational managers think is important. As a general rule, the
more closely numbers reflect what your customers value most, the better off those metrics will be.

2. Limit the number of metrics – Avoid information overload

In order to relay derive valuable actions from the metrics, one should implement no more than a handful (seven/eight) metrics at a given time. Getting too bogged down in measurements, one will lose
time and focus, employees will get confused, and upper managers may lose track of what you’re doing. This blinding by numbers can actually prevent progress.

3. Metrics must be defined

If it walks like a dog and barks like a dog, it’s a dog. A detailed description of the measurement process should be given in such a way that it can be easily understood. Generally called
Operational Definitions, these include a precise definition of the characteristic and how, specifically, data collectors are to measure the characteristic.

4. Visually represent the metrics

Data, by itself, can be overwhelming and difficult to analyze. Human brain works best by building patterns of the data captured. Graphical representations of data can highlight important aspects
within the data and assist the viewer in focusing on important items. In certain cases, visualization of information can assist the viewer in being more efficient with the analysis.

5. Metrics should have fast feedback loop

The measurement systems must provide feedback promptly, so you can identify problems and correct them as soon as possible. They should not be cumbersome or take a long time to yield data. Metrics
should be kept simple; avoid setting up complex measurements that are difficult to use. The information they provide should be used to take direct action. Metrics should not complicate operations
and create excessive overhead.

6. Ensure corrective action based on the metrics

It’s not enough to aim, you must hit. At the very outset, there should be thought given to the use of information gathered in addition to the metrics themselves. The process should ensure
that these KPIs/ metrics should provide the requisite feedback for the team to take corrective action as soon as possible.

7. Create an auditing process for your metrics

If you don’t ask the right questions, you won’t get the right answers. Since changing business conditions entail re-thinking your business strategy often, it is important to protect the
integrity of your metrics data.

8. Have cross functional metrics beyond the business unit numbers – Enterprise metrics

There is also a need to ensure that KPIs are not defined in only functional terms but also cross-business enterprise wide.

9. Perform sensitivity analyses on the metric

It is important to know the degree of importance of your business metrics. The data captured could have a linear or exponential influence on results. May be a scientific weighted average of
different metrics is need to removed certain management biases, like in projects prioritization.

Measure twice, cut once – some examples

Key Metrics for Inventory Management

Key Internet Metrics

Key Metrics for Customer Profitability