THE DATA ADMINISTRATION NEWSLETTER – TDAN.com
ROBERT S. SEINER – PUBLISHER

Being a Chicago native, it was with great anticipation that I returned home. The North Water area was a perfect setting for the TDWI spring conference where exhausted members could watch tour boats putter up and down the emerald green Chicago river as their cargo discovers Chicago’s architectural wonders. I wonder if Mayor Daley decided to permanently dye the river. From the patio level, during conference breaks you could see Millennium Park and Lake Michigan to the east and the fabulous Trump Tower going up floor by floor to the west. Da Bears! The Cubbies! The food! The Magnificent Mile! Navy Pier. Grant Park. This place is still alive! Just like I remember.

More on Chicago sites later, let’s get back to the conference. The theme for the spring conference had a decided business intelligence (BI) focus with an accent on business (read process, people and culture). I find this particularly satisfying. Those of you that have attended my courses know I get on my business process soapbox to tout the critical importance of understanding the business process in addition to the data. Not only did I feel at home in Chicago, but also right at home with the conference content! The conference drove home the importance of keeping our customers in mind. It’s not about the technology; it’s about encouraging the adoption of solutions by our customers. It’s about getting real return on investment.

People First!

David Wells had the opening keynote session on Monday, which was aptly titled People First – Creating Business Intelligence Culture. Dave set the tone for the Chicago conference: BI is for the people. If our customers don’t actually use the data warehouses and business intelligence solutions we build, all we have are expensive solutions looking for a return on investment. We need to put the people first!

Dave opened the session with snippets of the BI definition from a handful of industry thought leaders.

• Howard Dresner’s definition “… a set of concepts and methodologies to improve decision making in business through the use of facts and fact-based systems.”
  
  
• David Loshin’s definition “… the processes, technologies and tools needed to turn data into information, information into knowledge and knowledge into plans that drive profitable business actions. Business Intelligence encompasses data warehousing, business analytic tools and content and knowledge management.”
  
  
• Larissa Moss says, “BI is neither a product nor a system. It is an architecture and a collection of integrated operational as well as decision-support applications and databases that provide the business community easy access to business data.”

While these definitions all touch on necessary elements, they are not all encompassing. We need to add two additional elements that are central to the BI definition: “people” and “intelligence.” Dave’s refined definition: “Business intelligence is the ability of an organization or business to reason, plan, predict, solve problems, think abstractly, comprehend, innovate and learn in ways that increase organizational knowledge, inform decision processes, enable effective actions and help to establish and achieve business goals.” To create a BI culture, we need to put intelligence in context. Intellect, the root, originally came from Latin “intellectus,” which means understanding or discerning. Further extending intellect in context of business intelligence, it is the human capacity to reason, plan, solve problems, abstract, understand, innovate and learn. In 1993, Howard Gardner published a theory of multiple intelligences identifying seven core types: linguistic, logical-mathematical, spatial, bodily-kinesthetic, musical, naturalistic, interpersonal and intrapersonal. In the true nature of intelligence, Gardner is currently exploring the potential of an existential intelligence. Deep stuff, but what does this have to do with a BI culture?

The People Part:

• Fact vs. Perception – Humans use knowledge and experience during the thought processes, resulting in the formulation of perceptions, instincts, intuitions, beliefs and biases. Sometimes we confuse our perceptions with facts and our beliefs with universal truths.
  
  
• Capabilities and Talents – Gallup’s Strength Finder model identifies 34 categories of strength such as: adaptability, communication, connectedness, responsibility, inclusiveness and self-assurance. Capabilities and talents combine these strengths with interests. For example: A talented business analyst will combine analytical strength with curiosity to find business insight.
  
  
• Behaviors – Human behavior influences how our intellect manifests. How we think, act and work, as well as how we prefer to interact with people, processes and technology, are all behavioral influences.
  
  
• Culture – People create challenges for themselves and others through their interactions with other individuals, other members within their group and other groups. This interaction challenge is the seed of culture.

The Business Culture:

Dave discussed that culture encompasses eight critical areas of organization practices:

• Relationships – the form and manner in which people and organizational groups interact and conduct daily activities.
  
  
• Effectiveness – the capacity to produce meaningful and valuable results.
  
  
• Alignment – the degree to which parts of a system or organization are compatible, coordinated and cohesive.
  
  
• Accountability – responsibility assigned to someone for activities and their results.
  
  
• Responsibility – the social forces that bind an individual to his/her obligations.
  
  
• Commitment – the degree of dedication to a purpose and course of action.
  
  
• Change – the movement from one state to another. Change is constant, yet is met with resistance. So resistance to change is also a constant!
  
  
• Values – the characteristics that are most influential in shaping organizational behaviors.

The Business Intelligence (BI) Culture:

In order for BI to flourish, we must consciously establish core beliefs that promote fact-based analysis, information sharing and transparency. Howard Dresner says “To be effective with BI, organizations should have a culture that values fact-based analysis and the open sharing of information – i.e. transparency. This goes against human nature as most believe … that ‘knowledge is power’. Until we can overcome this natural tendency, an ‘Information Democracy’ cannot flourish.”

The challenge of the BI culture is that it is not an exact science. There are many ways to describe a good BI culture such as: fact-based and open sharing culture; culture of collaboration, control, cultivation and competence; analytic culture; culture of innovation and discovery; evidence-based and consensus-based culture; culture of measurement and accountability; competitive and learning culture; decision making and performance-based culture; customer-centric culture (internal and external customer); and continuous improvement of culture.

Dave’s point is that we can’t create culture. It already exists. We can learn to shape the culture into one of continual evolution toward a healthy BI culture. Culture is the collective beliefs, behaviors and assumptions of people groups. We are people. BI depends on us, thus we are all stewards of our BI culture.

Holy cow!

Making Sense of an Unstructured World!

I dropped in on Philip Russom’s class on unstructured data. Surprisingly, there’s a lot more unstructured data in your company than you would suspect. Philip shared some numbers from a recent TDWI survey. To my surprise, only 47% of enterprise data is considered structured, 22% is semi-structured and 31% is unstructured. Yet within the data warehouse, 77% of the data is from structured data, 14% from semi-structured and 9% from unstructured. That’s a lot of untapped information assets, especially for companies in industries needing to detect fraudulent activities, protect intellectual property and monitor for safety and security regulation compliance. The informal poll of Philip’s 70+ class participants confirmed the importance. Philip’s message was clear. Our current BI technologies don’t have the ability to issue unstructured queries. In order to meet the needs of our business, we need to develop solutions that turn the information embedded in unstructured data into something that is consumable by structured query language applications. SQL is still best to interact with database management systems. Philip provided a clearer understanding by putting unstructured data in context with content management, search technologies and BI analytics.

The Candy Man

If you crave chocolate, like to talk and want to learn, a Steve Hoberman class is for you. Steve rewards class participation by hurling Mars candy classics at you. Might want to bring your fielders’ glove!

Steve’s data modeling class is always a huge draw. There had to be 150 people in the class. You just can’t get away from the foundational building blocks, and this guy is the best in the industry to teach you. Being a data architect at heart, I can’t help but feel reassured that TDWI is still mentoring a new generation of data architects. Steve lays out the architecture, showing the importance of explicitly designing and implementing architectural components to enable the intake, integration, distribution and access functions. You also learn the importance of effective time-box data modeling techniques starting with gathering an inventory of business intelligence questions that need to be answered. Steve highlights the importance of business process knowledge. (Ahh, Steve and I share the soapbox!) The business intelligence questions provide the input to build a fact/qualifier matrix. (I see visions of Mrs. Crammer, my 3rd grade teacher, walking us through sentence diagramming, breaking the questions down into the verbs, subjects, articles and adjectives.) We’re diagramming business questions, transforming the verbs, subjects, adjectives and adverbs into facts, qualifiers and intersection points. Armed with the inventory of facts, qualifiers and interactions, we have our raw materials to construct dimensional data models to support the ability to answer those business intelligence questions. Steve introduces the concept of data archeology. We have to go find the operational data elements that will satisfy our hungry dimensional models. This is where the enterprise data warehouse (EDW) data model is applied. A well designed 3NF model for the EDW will help our solutions stand the test of time. We explored the interrelationships between enterprise logical models, enterprise data warehouse models and why we should design the enterprise data warehouse using relational modeling techniques and design most data marts with dimensional models. We learned how to use a combination of top-down analysis and design, to satisfy the dreamer in us, with bottom-up, to make something happen given what’s really available. Holy mackerel!

Exhibitionists Welcome!

One of my favorite parts of any TDWI world conference is hitting the exhibitor hall during the Tuesday and Wednesday lunch break. You owe it to yourself of hit all of the exhibitors. There are so many new products on the market covering the whole life cycle of enterprise data warehousing and business intelligence solution deployment and management. This is a one-stop shoppers dream. (The give-aways are cool!) You can gauge the evolution and innovation of the tools and products by walking the exhibitor hall. The mainstream players are always in full force showing their support for the TDWI value proposition. The data warehouse appliance market is getting interesting. As I strolled through the hall, I felt a sense of pride for these providers. The likes of DATAllegro, Dataupia, Netezza and ParAccel are ushering a new era of data warehousing solutions.

The open source tool providers are also of interest. These tools such as JasperSoft, Pentaho and Talend continue to get more robust, and their value proposition warrants a serious look. They have a compelling total cost of ownership story to tell.

Tell Me What You Think

Another reason for enjoying time in the exhibitor hall is because it allows me to do two of my favorite things: eat and visit with the attending members, exhibitors and instructors. Lunchtime during the exhibition days (Tuesday and Wednesday) provides a great opportunity to network. Lunch is setup buffet style, and large round-top tables are set up around the perimeter of the hall. Grab yourself some grub, find a table with one or two openings and ask to join the group. Do not find an empty table. That’s not in the spirit of the TDWI meet and greet networking mission. I always ask my lunch mates how they like the conference. Now I read those “how to talk to your kids” books, so I don’t let my captive interviewee off that easy. I am ready with my follow-up questions to the anticipated “I really like it. Has a good beat!” response. I ask them to share one highlight and one lowlight about the conference. I do warn them they may see their feedback quoted (anonymously of course). Here’s a few of the comments I received:

“It is awesome to take a full day to just focus and get my head around one topic. My lowlight? Well I really wanted to get into Stephen Brobst’s Sunday class and now I’m in the dog house for attending a conference on Mother’s Day!”

Hmm, note to event planners. We don’t want to compromise our members’ domestic tranquility!

“For me the value of this conference is the ability to meet with so many of the industry thought leaders in one place at one time. I have been in data warehousing for some time now. I need instructors that provide the practitioner’s perspective. This is where the guru sessions have been great. My low light? I need more guru sessions!”

 “Good non-biased instructors. The TDWI model needs to be replicated…the United States could get back into a position of technology and thought leadership.”

Power, Politics & Partnership

So after a nice lunch and a stroll along the Chicago River, I thought I would sneak in to relax and watch Maureen Clarry and Lorna Rickard in action. Fat chance! You don’t sneak into their class, and you certainly don’t get to observe. I opened the door to see groups of people huddled together. They all look up at me, total silence. I smile and try to fade into the velvet wallpaper in the back of the room. Next thing I know, I’m a member of the “Tops” and sent off to participate in Day 3 in the life of the Tops. We’re the executives of the “Firm” where shoes have been taken away from all employees; but if they work hard, the Tops might consider issuing slippers even though the Department of Labor levies a healthy tax on slippers. In a matter of ten minutes, I realize I’m not cutting the grade as a Top at the firm. My clients are in need, the Department of Labor wants to perform a process improvement audit, my managers (the “Middles”) are in a state of chaos and my employees (the “Bottoms”) are in need of a raise so they can buy firewood to warm their frostbitten toes. This course really drives home the need to step into the other person’s shoes in order to develop effective strategies that will garner the kind of collaboration and teamwork proven critical to ensure success. It helps you see that by giving up your personal needs, you can contribute to a higher level of enterprise-wide effectiveness. Customers learned to give up the need to be right. Tops learned to give up the need to control. Middles learned to give up the need to be popular. Bottoms learned to give up the need to be the victim. This really resonates with my experience in the data warehousing field where the teams are usually a result of a matrix management organization and need to get cooperation from the application support teams and IT infrastructure teams as well as get explicit sponsorship and participation from the business executives and managers.

Where in the World is “Data”?

We’re in David Loshin’s class to learn data requirements gathering techniques with 49 fellow TDWI attendees. Wait, what’s this? David has that fact/qualifier matrix up there as one of our end goals! This thing must really be an important design artifact. (I gotta tell my boss and my project manager about it.) We learn that getting a well conceived fact/qualifier matrix built requires an enterprise-wide semantic understanding of the data elements. David tells us (my paraphrase), “Don’t underestimate the impact of semantics. For example: Last night I called my wife. I dialed her mobile phone. When was the last time you ‘dialed’ a phone number? It’s all in the semantics.” Point being, if we want to promote consistency across projects and the enterprise, we need to invest in the development of an enterprise glossary of information terms complete with semantic cross-references. David shares that business process models are a valuable tool for data discovery. (Note to self: need a bigger soap-box!) The information flows between business processes provide insight to the data elements of interest for reporting performance about those processes. We can start to create conceptual data models to support the business process models, and we can start to populate that enterprise data element glossary.

We also learned where data profiling is effectively used during data discovery. Once we find our application systems and look at their metadata, we can use data profiling to better understand the application databases by inspecting the actual content. Surprise! The application system data structures don’t always reflect the actual use of those applications. The data profiling discoveries provide insight to data quality rules against which we might want to subject the data. Understanding the potential data quality concerns with data is another valuable discovery process. Ask the business what issues they have with a data element in context of performing their job. Ask the questions in context of their production of the data and the consumption of the data. Look for “fit for use” conformity rules.

I guess there’s a little bit more to this data warehouse data modeling than drawing those pictures with boxes and lines!

Tool Talk!

TDWI’s mission is to deliver world-class thought leadership, education and research intelligence in the data warehouse and business intelligence industry. TDWI is vendor agnostic. Their instructors are charged to deliver instructional and educational UNBIASED courseware. TDWI has two sets of customers. The TDWI members and the exhibitors. In another perspective, TDWI sees the members as a customer of goods and services not only of TDWI, but also of the suppliers of the goods and services. (The exhibitors). Tool Talk is an experiment to facilitate interaction between the customers and suppliers. This night school session was set up as a panel discussion, moderated by yours truly. Six BI solution providers accepted the panel invitation representing Business Objects, a SAP Company; Cognos, an IBM Company; Information Builders; MicroStrategy; Oracle; and SAS. The objective of Tool Talk is to provide a forum for the members to ask the vendors some burning “how to” questions. We allocated time for up to 6 questions from the audience and time-boxed the responses from the panelists. Two panelists responded to each question with one taking the primary response and the other providing response/rebuttal/augmentation. Each vendor was given an opportunity in the primary respondent role and the rebuttal/augmentation respondent role. The panelists were held accountable to respond to the question without any sort of sales pitch. Two of the questions are highlighted below.

Question:

Given, the continual growth of the details available in our enterprise data warehouses, what are you doing to address the performance of your product when the BI user wants to process and reprocess thousands of rows of data in their reports?

Responses:

• 3 of the vendors responded like a consultant. They challenged the validity of the end-user truly crunching thousands and thousands of lines of detail on a report. They offered instead to take a closer look at the true business intelligence requirements to find a more effective means to display the answers to the underlying business questions and enable the discovery of why the answer is as it is. The panelists suggested digging into the business processes to get a better appreciation of what the end consumer is trying to measure and the activities they need to execute when performance is not meeting expectation. The panelists further offered that upon discovering the root cause, the solution should enable the ability to push the drill-down details to appropriate data mining and analytic functions.
  
  
• 3 of the vendors responded like a vendor selling software, indicating that their product was designed to take advantage of today’s technology including 64-bit architecture, which opens the door for pushing more of that detail into RAM.

My readout:

• I appreciated the thoughtfulness that went into the responses from the vendors that wanted to dig into the proper implementation of the business solution. This supports the goal of nurturing our customers to move up the BI maturity curve.

Question:

When will we see the day where you all support an open metadata information exchange platform so we can bring critical enterprise-wide metadata together?

Responses:

• All 6 vendors responded that we’ll not be seeing anything like this in the foreseeable future. Their metadata is their secret sauce.

My readout:

• The product providers need to rethink their position on offering a vehicle to expose the metadata that is important to their customers. We don’t need to be exposed to the secret sauce of their product, but we do need the ability to see our metadata. Definitions of the information objects, the calculations and rules for use of those objects, the reports, dashboards, scorecards, modeling programs that deliver information and the end-users that use those information delivery vehicles.

Self-Serving or Self-Service BI?

Wayne’s message, The Myth of Self-Service BI, was the perfect topic for a Thursday morning keynote address. The place was packed. Come Thursday morning, people are starting to look like deer in the headlights. Many of them have been drinking from a fire-hose for the last 4-5 days, but self-service BI is a hot topic these days and people wanted to hear what Wayne had up his sleeve. Wayne unveiled his MAD-PAD framework. (I had visions of Alfred E. Newman doodling on a tablet PC dancing about my head.)

We took a quick stroll down memory lane recalling how, in the mid-1990s, the dawn of data warehousing freed our customers from IT prison. The celebration didn’t last long. Soon those sovereign customers were back on the chain gang waiting for their reports from the data warehouse. IT, realizing they couldn’t possibly get the budget to staff for this increased demand, needed a solution so IT removed itself as the bottleneck and created the concept of self-service BI. Let the end consumers create their own reports. Empower the user! Just as Marie Antoinette was oblivious of the French citizens’ needs, so was IT being oblivious of the business intelligence needs of the consumer. Let the end-consumers eat cake! (Ron Santo would say, “Let them eat hot dogs!”) The IT backlog is gone, liberating us once again. Oops, the problem is most users can’t create reports. Even if they do know how to create reports, they don’t know what they want on the reports or don’t know where to get it or, worse still, they guess and create a hideous monster report. The users need help. IT is back in the bottleneck as demand for support mushrooms.

Wayne asks, “Do users really want to be retrieving and formatting data, or do they want to monitor and analyze information and take action?” When you consider the ole 80/20 rule where 80% of users are casual, and you consider the types of reports needed, nowhere do you see the casual user as an author. The business, being weary of the backlog queue, hires the “power user” who is compensated to produce reports. The result? Report chaos! Tens of thousands of reports and “spread marts” proliferate across the enterprise. We have 20% of the users creating lots of reports with different renderings of “truth,” and 80% of the users can’t find the right report with the right results. The power users are dimming the lights with runaway queries, and the casual users can’t get the one report they did find to even run. (Cake tastes good, but too much cake is not good for you!) You know you’re in the midst of shifting the burden when the harder you try to fix the problem, the worse it gets. You can address the IT report backlog by implementing self-serve BI. Self-serve BI provides short-term relief, but long-term effects will result in increased demand for IT support. Be wary of total reliance on self-service BI.

Enter MAD-PAD (Hello Alfred!): Wayne introduced the concept of tailored delivery, an interactive information sandbox, as a more elegant means to deliver self-service BI. Tailored delivery consists of a layered information delivery system combined with a set of performance dashboards and/or parameterized reports tailored to a specific end-consumer group and the ability for the individuals to personalize their dashboards or parameterized reports. The tailored delivery solution contains roughly 20 dimensions and 12 metrics. The users experience “ad hoc” interaction, which answers 60-80% of their information needs. Each tailored delivery solution can replace dozens or even hundreds of self-service reports. The sandbox provides functionality to support the end-consumers (managers, analysts and workers) with the ability to monitor, analyze and drill (MAD) as they consume graphical data down through summarized data into detailed data. Adjacent applications can be integrated into the MAD sandbox. These applications would support the planning, advanced analytics and decide & act (PAD) functions.

So the concept of self-service BI is valid. It is the IT implementation that has gone awry due to IT’s over-exuberance to empower users. The tailored delivery sandbox provides a means to bring balance to the self-service model. IT cannot completely abdicate responsibility when rolling out self-service BI. Tailored BI is a more elegant way to implement self-service. It is the key to pervasive BI as long as IT strives to know their users and their roles, including the ad hoc role. To implement effective PAD applications, we need to understand the business processes. (Good thing I got that bigger soap box, eh?)

My Kinda Town!

Did I mention how much I like to eat? If you share my passion for tucker, Chicago is your kinda town too. For lunch, you can’t beat the gastronomic delights such as the original Chicago style “Garden on a Bun” hot dogs and Italian Beefs dripping with au jus. “Garden on the Bun” was created by European immigrants in the nineteenth century and elaborated upon by succeeding generations. The subject of many jokes, including the name itself, the hot dog represents a social and cultural history of America. Nowhere are these themes better seen than in Chicago’s hot dog stands.

For dinner: My first night in Chicago, I dug into the thick morass of cheese, pepperoni, mushroom, green pepper, onion and tomato sauce sitting on top a perfectly flavored sour dough crust. The Sheraton is just a stone’s throw from Rush Street, the location of Giordono’s Pizzeria. The Chicago classic deep dish pizza experience is a must for any visitor. (This humble grabowski still finds Lou Malnati’s to be the city’s finest deep dish, but the large Giordano’s did indeed satisfy.) Dinner the second night was in Greek Town, a short cab ride from the hotel. Santorini’s was our destination. We started with appetizers of saganaki, dolmades, loukaniko and fried kalamari. For the main course, we went family style, sharing some classic dishes like lamb stew and sea bass Greek style. We wrapped up the meal with thick quarters of fresh watermelon and semi-sweetened Greek coffee. The third evening on the town, we headed to Gene & Georgetti’s Steak House. This is Chicago’s oldest steakhouse founded in 1941 by Gene Michelotti and his partner Alfredo Federighi, who was nicknamed “Georgetti” after a famous Italian cyclist. We were greeted in classic gruff, Italian, Chicago style by the host: “Hey, how ya do’n. Yeah, follow me.” Family style appetizers of fried ravioli, shrimp, calamari and sausage, followed by absolutely the best 14 oz. filet mignon I’ve had with sides of tomatoes, mashed potatoes, asparagus and sautéed mushrooms was all wrapped up with a resounding chorus of “Happy Birthday” by “da waiters.” (Nice bunch of guys.

A Social Comment

Having turned the half-century mark during this conference and being on the tail end of the baby boomer era, I notice things that make a difference for our aging generation. I take my hat off to Paul Kautza and the staff at TDWI. What a beautiful community service TDWI provides to our senior citizens. The classroom monitor services are provided by our senior citizens. A handful of them tour with the conference! You couldn’t find a better group of room monitors. They are bright and cheery and always ready to lend a helping hand to the members and instructors to ensure a pleasant experience for all. Nice touch!

Up Next!

I’m really looking forward to the next conference in San Diego. The keynote speakers will be Howard Dresner and Bill Schmarzo. The Executive Summit, a special track geared for the Vice Presidents to network with their peers and industry thought leaders, will also be running. There’s a great line-up of instructors and sessions. Let’s not forget the gorgeous weather! Perhaps we’ll have a chance to visit at the exhibit hall!