Becoming a Data-Driven Company

Data-driven companies are more profitable than their competitors and outperform them with regards to the acquiring and retaining of customers [Morris18]. This is a one sentence advertising slogan that explains why so many companies transform their organization and invest into building and expanding their data management and analytics capabilities.

Understanding Data Driven

“Data-driven company” is an intuitive concept.

Companies and organizations want to base their decisions on actual data and on insights derived from their data. Gone are the days of decision making based on gut instinct. Data-driven companies and organizations let the data speak for itself. And we might not even notice this, because “a key characteristic of data-driven culture is using data in a pervasive way” [Gartner 15].

While the definition is intuitive, it falls short on helping CIOs and their IT departments to find an effective understanding of the data. Researchers, however, developed a framework for data-driven decision making [Mandinach06]. Though, coming from a completely different field—US education and policy—it perfectly highlights the challenge for IT departments. It is based on three concepts: data, information, and knowledge (Figure 1). Data is the raw form. It does not carry any meaning and may or may not be useful. Information is contextualized data carrying a meaning. Knowledge is a collection of information that guides decision making.

Some examples from corporate information systems or the Internet of Things (IoT) illustrate these concepts more in-depth for our context of data driven organizations. A stream itself, consisting of 1s and 0s, is data. Strings and numbers stored in database tables are data. Giving context is what transforms data into information. The stream of 1s and 0s is an audio stream monitoring an engine. One column with strings in the database stores customer names, the other employee names. The various columns with numbers represent how much the customers ordered or how profitable they are. This is information because now, the 1s and 0s and the digits and characters have a meaning that we can understand. This information turns into knowledge if it helps guide decisions. Do we notice irregularities in the audio stream of the engine that might indicates technical issues? Who are our Top-100 customers? Which customers did not order for more than a year?

Such knowledge helps humans making better decisions. However, it is not a replacement of human judgement. Humans (usually employees outside of the IT department) decide whether to call the top customers or inactive customers to push sales or whether to switch off an engine. Certainly, many decisions are based on rules that can be written down. These rules are then translated into code. The decision making becomes automated. However, the base for the decision making is still the human judgment and decision. It is just automated using computers, as you might know from fully automated credit scoring applications.

The shift towards knowledge generation has implications for IT departments and especially for their Business Intelligence and Data Warehousing services. While at the forefront for years, many are now under pressure. Their classic service—consolidating data, understanding their exact meaning, generation of long lists—is still essential, but not ready for an agile fast-moving organization or simply not comprehensive enough. Our framework in Figure 1 provides the explanation. Long lists of consolidated data are information, not knowledge. Everything is still crucial, but there is a gap between user expectation and the actual service. The business needs and wants knowledge. Not every IT department is ready (and has the funding) for that. Still, the market trend shows us two popular ways how IT departments can enable knowledge generation:

1. Provide tools and data for the end users such that they can derive the needed knowledge themselves by exploring “their” data. This is termed self-service business intelligence. Examples are Qlik or Microsoft Power BI.
2. Provide a service to build and run analytics and machine learning models. IT experts use advanced computer science and statistics to derive insights humans cannot derive. TensforFlow or R are well known technologies in this area.

The Business Value of IT Projects in the Context of Data Driven Organizations

Certainly, most managers know that their organization must become data-driven. It is also clear that the IT department must shift and expand its focus (which requires funding). Still, it is unlikely that IT projects without a clear business case will receive funding. This is often a hurdle for highly innovative projects. Here, it helps to look at economic research to remember the four ways how IT investments can generate value [Gregor06].

1. Informational benefits: delivering information to the decision makers
2. Transactional benefits: improving daily operation and cutting costs
3. Strategic benefits changes:changing how the company competes in the market and/or the nature of its products.
4. Transformational benefits: (referring to future benefits) using the new tool for additional optimizations once the business gets used to it.

A recent study looked on concrete examples for all four areas and verified their relevance with French companies of various sizes [Elisabetta18]. The study focused on Big Data, a field closely related to data driven organizations. They identified enhancing employee productivity to be the most relevant transactional benefit in all considered industry sectors, whereas reducing operating costs was second. Regarding strategic benefits, providing better products or services was on top, followed by aligning IT with a business strategy and enabling a quicker response to change. In the area of transformational benefits, expanding capabilities was most important. It is explained that new employees join the organization and integrate additional data sources within a short time frame. In the area of informational benefits, they list proving management data, improving data accuracy, providing data in more useable formats, enabling easier access to data, and enabling faster access to data.

There are two key points when applying the insights for your business case for IT projects. First, the area was close, but not exactly the same. They are a good inspiration, but not god’s word. Second, if you have a lot of experience with company politics, it is obvious that not all benefit types are well suited for each organization:

• Certain benefits are easy to quantify. This applies especially to transactional benefits, but also to others such as concrete benefits (data accuracy or enhancing capabilities). They are perfect even for companies driven mostly by financial numbers with strong influence of controllers.
• Some benefits are difficult to quantify. This holds true for many strategic and transformational benefits. They are well suited to convince visionary and innovation-driven decision makers.
• Informational benefits can be the easiest to fund. When a manager believes there is a need for data, they will fund an initiative if it fits into the budget.

However, one underlying truth should never be ignored. The benefit is usually on the business, not the IT side. No talented IT expert can switch a company to become data driven. Thus, we have to have a closer look on the interplay of the IT department and the business.

The Roles of the Business and the IT

Becoming data driven is a joint effort of the IT department and the business. When we talk about organizational changes that are enabled by the IT department, this requires that the business also invests time [Gregor06]. When we take a high-level view of the project phases, this becomes obvious (Figure 2).

The starting point is always a business question/problem—often inspired by strategic goals on the business side. The next step is to define the project scope. This is a joint activity of the business and the IT department. They answer questions such as which exact business problem to address, the technologies to be used, or the business case. Then, the IT department will work on collecting the needed data and build. For example, an analytics or machine learning model is selected to answer the business question. In parallel, the business works on adapting its processes for the new technology. When both are ready, the business begins working with the new business processes.

This illustrates that the IT department has to rollout a comprehensive solution immediately useful for business users. In most cases, there is no need for a super-tuned algorithm winning academic awards. Instead, the data scientists and engineers must deliver a complete end-to-end solution taking the needed data, generating a model, applying the model, and delivering results for daily decision by the business in a user-friendly way.

So, what is the conclusion? How do IT departments have to act to be ready for a data driven organization? The following three statements summarize the main points:

1. The IT department has to shift its focus on knowledge generation, i.e., on providing better input for human decision making—or by automating decision making.
2. The four potential benefit dimensions hold true as well in the age of data driven companies. There are informational, strategic, transactional, and transformational benefits.
3. The IT department is an enabler. The IT enables the business to move to data-driven decision making. Thus, projects can only succeed with the buy-in from the business. They have to be willing to invest time and redesign their processes.

[Elisabetta18] R. Elisabetta: Big data technologies: An empirical investigation on their adoption, benefits and risks for companies, International Journal of Information Management, 81(1), 187-195

[Gartner15] E. Dunlea: The Key to Establishing a Data-Driven Culture, https://www.gartner.com/smarterwithgartner/the-key-to-establishing-a-data-driven-culture/, last retrieved April 6^th, 2020

[Gregor06] Sh. Gregor et al.: The transformational dimension in the realization of business value from information technology, The Journal of Strategic Information Systems, Volume 15, Issue 3, September 2006, Pages 249-270

[Mandinach06] E. B. Mandinach et al.: A theoretical & framework for Data-Driven Decision Making, annual AERA meeting, San Francisco, April 9th, 2006

[Morris18] T. Morris: 6 Competitive Advantages of Data-driven Organizations, Micro Strategy Block, June 15th, 2018, https://www.microstrategy.com/us/resources/blog/bi-trends/6-competitive-advantages-of-data-driven-organizati, last retrieved April 20^th, 2020