Published in TDAN.com July 2001
Business Challenges
It is said, “the only constant in life is change”. Never has this old saying been more applicable than in the corporations that are competing in today’s marketplace. Business is evolving in
ways that were unforeseen 20 years ago. Running and managing a global company is more difficult than at any other point in time. At one time companies just had to worry about producing a quality
product, in an efficient manner, appropriately priced, and properly distributing it to their customer base, however in today’s marketplace this is not enough.
Emergence Of One-To-One Marketing
Growing consumer dissatisfaction and expectations, and global competition are quickly driving corporations to move from the mass production concept to a one-to-one marketing strategy. The mass
production concept was one of “build it and they will come”. The one-to-one marketing strategy simply states “create products/services that satisfy your customer’s individual needs”. For
example, a friend of mine recently moved into a new area of the country. She went to her local grocer to purchase various food items. This grocer, like most in the country provides coupons to aid
in enticing the customer to come back to the store. They provide different coupons depending on what the individual purchased. As my friend typically does she purchased her favorite brand of
waffles (Brand “A”), that she has been eating exclusively for years. Upon checkout she received a coupon for waffles, unfortunately the coupon was for Brand “B” as oppose to her favorite, that
she has been buying for the last 20 years. If only the store had insight into her buying habits they would have understood that Brand “A” is the only kind of waffles she is going to buy.
One-to-one marketing looks to understand our individual buying trends and then market appropriately to them. Making the transformation to this new one-to-one marketing approach is forcing CIOs
(chief technology officers) and IT (information technology) directors to radically change their IT strategy to support this strategy.
Evolving Distribution Channels
At one time it was common to have a person come to your door asking if you would like to purchase an encyclopedia set. These encyclopedia sets typically encompassed 20 – 30 very thick, and
heavy hard copy books. In addition, to the initial purchase each year the consumer would have to purchase an additional book that contained the updates that would occur after the previous
year’s publication. In today’s marketplace the door-to-door salesman has been replaced by the internet, where you can electronically purchase your encyclopedia set on a few CD ROMS
which contain illustrated moving pictures of many of the topics covered in the CDs. In addition, there is no longer a need to purchase updates to the encyclopedia set. Instead there is an
accompanying website which that keeps the information fresh, and more timely than ever before.
Global Marketplace
Previously a company could be assured of having a solid, and stable market base just because of their geographic location. With each technological advance occurring on an almost monthly basis our
world is becoming more and more accessible. Just fairly recently there have been such technical advances as cellular phones, global pagers, video conferencing, the internet, email, and a highly
advanced package delivery (next-day air, etc). All of this has allowed companies to market their products quickly and cost-effectively to just about any place in the world.
Now that we’ve reviewed the reasons why business is changing at such a rapid pace, next month we will examine the technical challenges facing CIOs and IT directors in assisting their
companies in satisfying the needs of these business challenges.
In order to fully understand meta data’s value it’s imperative to identify the changes impacting businesses today. Last month we examined the three key areas impacting today’s
corporate landscape. These three areas are:
- Emergence Of One-To-One Marketing
- Evolving Distribution Channels (internet)
- Global Marketplace
Technical Challenges
Poor Systems Integration And Adaptability
The days of building an operational system and using it for 10 – 20 years, without major enhancements (typically complete rewrites), are over. In order for companies to adapt to
market’s rapidly changing landscape the enterprise’s data processing systems must be more flexible and robust than ever before.
Many to our current systems have been built as “stovepipe” applications. Meaning that they do not communicate easily with other corporate systems. Moreover, these “stovepipe” applications form
their own system “islands” with their own hardware platforms, databases, and development languages. Corporations are demanding new systems changes at an astounding rate and unfortunately these
old “legacy” systems do not adapt well to change. This is most clearly evident with the Y2K problem that has haunted most every company in the world. If we stop and think about it Y2K is merely a
date field that has been designed to hold a two-digit year, instead of a four-digit year. When the problem is considered in these terms is doesn’t appear to be very significant. Of course we
know that the Y2K problem is very significant as the code that manipulates the date field is very convoluted, difficult to identify, and hard to locate.
For those that believe that the need to make global systems changes end with Y2K keep in mind that immediately, following the Y2K will be the new standardized European currency (EURO) challenge.
While the EURO will significantly impact all corporations, financial institutions will be most effected.
Along with the rapid change in business needs there is an equally rapid change in the technology that fuels systems development. As a result, corporations are working feverishly to keep their
company’s systems up-to-speed with the extreme rate of change in technology. It seems that weekly there is a new software application or piece of hardware that would aid a company in meeting
their corporate goals. This situation is certainly not going to slow down in the upcoming years. Corporations are demanding faster hardware, and more software functionality and integration to gain
a corporate advantage.
Systems Must Deliver Better Business Value
Another challenge facing companies today is getting their systems to meet the needs of their business users in a manner and language in which the business users understand. As an industry we as IT
professionals are still not sufficiently meeting the needs of our business users. One recent survey asked CEO if their IT systems were meeting the needs of their business. Over 84% of the CEO felt
that their systems do not. As an IT professional this is a statistic that cannot go unnoticed. Instead of designing systems that speak to them in business terms, the systems we are building still
communicate to their users in IT terms. For example, I experienced one such system that would show the user a report called XC001AB, which is a report that shows product sales, by region, by
marketing campaign, over time. Clearly a marketing analyst would much rather see this report entitled “product sales, by region, by marketing campaign, over time” as oppose to XC001AB. This small
example illustrates why so very few senior executives believe that their corporation’s Information Technology strategy is well-integrated with their business strategy. The good news is that
the vast majority of these senior executives do believe that their systems are critical to the success of their business.
Decision Support Moves to the Forefront
Most companies have come to the realization that a decision support system is critical to their enterprise goals. Operational systems are typically designed to manage products. These systems have
evolved over the years to produce, deliver, and invoice products/services. . Unfortunately, when remedial questions are posed:
- “Who are our most profitable customers?”
- “Which segment of our market offers the greatest future potential profit?”
- “Which of our products are complimentary (market basket analysis)?”
- “Which competitors pose the greatest threat to our existence?”
- “Which product/service is providing the greatest value to our customers?”
These operational systems cannot easily or quickly provide answers to such fundamental questions that are redefining the world of business. The emergence of decision support looks to transition
these legacy (operational) systems to decision support systems (DSS) that help companies answer these questions, which support a one-to-one marketing future. DSS system is designed to manage
customers as oppose to products. DSS systems are build to handle the “strategic” questions that a company’s key decision-makers need to answer. Moreover, companies now understand that
without a DSS system that they will not be able to compete effectively enough to survive in tomorrow’s marketplace. In fact I believe the in the near future, 5 – 10 years that decision
support will be a key component to any major corporate IT initiative. It will no longer be a decision of “if” we should build a DSS system, but “how” quickly we can build them.
DSS Challenges
The construction of decision support systems to aid in the strategic planning and decision making process. The task of replacing old operational systems is being impeded by out dated systems
designed to manage products, and not a company’s customer base. These “operational” systems have evolved over the years to produce, deliver, and invoice products/services. Unfortunately,
when a remedial question is posed “who are our most profitable customers?” These systems cannot easily or quickly provide answers to such fundamental questions that are redefining the world of
business.
It’s easy to see the need for DSS systems, unfortunately the task of implementing these systems is anything but easy. The challenges for implementing a DSS system (also know as a data
warehouse) come from both the business and technical arenas.
Business Challenges
The most common cause for DSS project failure is that once the systems is built it didn’t meet the business objectives of the organization. Data warehouses that don’t satisfy the
business user’s decision support needs are not accessed and eventually die.
Clear business objectives can be clearly defined and are measurable. This activity is critical since once the DSS project is completed the management team will have to justify the expenditures.
Moreover, it’s important to understand that a data warehouse is NOT a project, it is a process. Data warehouses are organic in nature. They grow very fast and in directions that are very
difficult to anticipate. Most warehouses double in size and in the number of users in their first year of production. Once a cost justification can be quantified for the initial release the process
for gaining funding for the follow up releases is greatly simplified.
From a political perspective an enterprise data warehouse requires consent and commitment from all of the key departments within a corporation. DSS systems pull data from all of the key operational
systems from across the enterprise and looks to create an integrated view of the corporation. This challenge is particularly difficult since rarely do separate departments agree on what that view
should look like. As a result, the political aspects of implementing these systems are particularly challenging.
Technical Challenges
DSS projects technically stretch an organization in ways unlike that of traditional system projects. Typically a data warehouse will source data from most, if not all of the key operational systems
within a company. The task for integrating all of this data from across the enterprise is considerable and requires the largest effort of any project activity. For example, it is probable that a
company would want to store customer data in the data warehouse. More than likely there is customer data in several of the firm’s operational systems. All of this dispersed customer data has
to be integrated and cleansed before it can be loaded into the data warehouse. The process for integrating the data is complicated since it takes a good deal of knowledge on the data in order to
integrate it.
DSS systems tend to store large quantities of data. It is even becoming somewhat common to see one or more terabytes of data being stored and accessed. By adding massive amounts of data into the
equation the points of failure increase significantly. Moreover, large volumes of data push the envelope of the database management system (DBMS), middleware, hardware, and could force developers
into using parallel development techniques, if massively parallel processing (MPP) architecture is needed. Keep in mind the answer to many of these challenges comes in the form of a hefty price
tag. As a result, adding the dimension of size can be painful and costly.
Many legacy systems contain redundant or inaccurate data. This lack of data quality in the operational systems has caused more than one decision support effort to fail. As a result, this
operational data must be cleaned before it is loaded into the data warehouse to ensure its usability. Metadata is critical for monitoring and improving the quality of the data coming from the
operational systems. Metadata tracks the number of errors that occurred during each data warehouse load run and can report when certain error thresholds are reached. In addition, the DSS data
quality metrics should be stored over the history of the DSS system. This allows corporations to monitor their data quality over time.
Meta Data Solutions
While these challenges are daunting indeed, corporate executives are discovering answers to them in the form of meta data. This realization is helping fuel the global interest meta data. The
functionality that a meta data repository provides to address these challenges include:
Impact Analysis
A meta data repository significantly reduces the costs of development and the time frame needed to do it in by allowing the IT (information technology) development team to run technical impact
analysis reports across all corporate systems stored in the meta data repository. These impact analysis reports significantly aids the design analyst as they examine the impact of proposed changes
into the DSS (decision support system) environment. For example, let’s suppose that a company was going to reorganize their geographic locations. Currently Mexico and Canada sales are grouped
with U.S. sales and the company’s management decides that they need to separate these sales numbers. The meta data repository would allow the development team to generate a impact analysis
report showing all systems and programs that use geographic regions. This benefits the development team by minimizing the costs of the system enhancement and helps to reduce the propensity of new
development errors.
Anyone that doubts the value of this analysis need look no further than the Y2K situation. What was Y2K? In the early years of software development, programmers stored the year portion of a date in
a field that could only hold a two-digit number (e.g. 90) instead of a field that could hold a four-digit number (e.g. 1990). As a result, many information technology systems in businesses today
will not function properly with dates beyond December 31, 1999. If we stop and think about it, Y2K was merely a date field that had been designed to hold a two-digit year, instead of a four-digit
year, therefore this was just a change in a field’s length. When the problem is considered in these terms is doesn’t appear to be highly significant. So how significant was this
problem? Federal Reserve Chairman Alan Greenspan estimates that several hundred billion dollars was spent trying to fix the computer glitch before the turn of the century. It would have been
interesting to see the ramifications of this situation for companies with a meta data repository that would allow them to run these types of impact analysis reports. It certainly would have
drastically reduced the impact of the Y2K system change by making our systems much more flexible.
Transforming Data Into Knowledge
The reason we exist as IT professionals is to meet the informational needs of our business users. As we discussed in the August column our current systems are falling well short of meeting these
needs. A recent survey asked CEOs “Do you feel your IT systems meet the needs of your business”. 84% of these CEOs felt that their IT systems did not meet the needs of their business. This number
is staggering and sends a clear message to all CIOs and IT directors. We as IT professionals must do a significantly better job at fulfilling the needs of our business executives, because they are
demanding us to pay greater attention to the value IT brings to the business.
One of the reasons this is occurring is that instead of designing systems that speak to our business users in the business terms that they are familiar with, we have built systems that communicate
to them in IT terms. Meta data addresses this situation as it looks to provide a semantic layer between our IT systems and our business users. In simple terms meta data looks to translate the
technical terminology into business terms that the users are familiar. For example, when a business user is looking at profitability numbers it would be very valuable to them to understand the
formula used to calculate the numbers and what systems were sourced to provide the data for the formula.
Data Quality
Many legacy systems contain redundant or inaccurate data. This lack of data quality in the operational systems has caused more than one decision support effort to fail. As a result, this
operational data must be cleaned before it is loaded into the data warehouse to ensure its usability. Meta data is critical for monitoring and improving the quality of the data coming from the
operational systems. Meta data tracks the number of errors that occurred during each data warehouse load run and can report when certain error thresholds are reached. In addition, the DSS data
quality metrics should be stored over the history of the DSS system. This allows corporations to monitor their data quality over time.
