Embracing “Data Town Plans” as a Bridge Between the Business and IT (Part 1)

Imagine building a new city without a “town plan,” yet some people build information technology (IT) solutions without the data equivalent.  

The Age-Old Challenge of “The Business” vs. IT 

IT often throws around terms such as data sharing, data integration, interoperability (between stakeholders, both within a single organization and further afield), or, heaven help us, data governance. But let’s be realistic. Such language probably won’t light the fires of passion amongst many less-technical business leaders. 

These lofty and admirable goals of data folk seem to be minefields littered with the bodies of well-intentioned initiatives. But does it have to be that way? For a number of organizations, creating a “data town plan” has proven to be the key to helping “business” management to articulate their vision for how they want to see their data (note the emphasis I have placed on the business), and in a manner that fuels IT delivery of valuable (and valued) solutions. 

Before we look at the idea of “data town planning,” let’s take a brief sideways step and consider a related issue — an observation that, over the years, IT sometimes has struggled to accommodate holistic enterprise-wide thinking while delivering bite-sized elements of the total solution. Then we’ll come back to “town planning.” 

Another Age-Old Challenge? “Think Big” vs. “Think Small” 

This article is intended to focus on what the business needs so that it can leverage value from its data; I’d rather avoid getting into heated debates from the technology teams. Nonetheless, there are real risks of flavor-of-the-month IT delivery approaches pushing themselves into the spotlight as some silver bullet. So, I’m going to take a risk and dance in the minefield! (Yes, lots of mixed metaphors — sorry.) 

Perhaps you’ve come across scenarios where businesses reinvent themselves around think-big “centralization” so that they can benefit from things like economies of scale and standardization? Some years later they realize that they have become top-heavy and are stifling innovation, so they change and go for think-small decentralisation. Oops, now they are experiencing pain from the loss of centralization benefits, so they flip/flop to and fro between centralization and decentralization. 

Before we poke sticks at “management,” we maybe need to eat humble pie and realize that IT has its own flip/flop think-big versus think-small issues. I’ll be brief, but perhaps say enough to make a few points: 

• In the 1970s, lots of IT systems were developed in isolation. That’s small thinking. Data sharing emerged as a problem, so some brave souls in the 1980s tried to think big and develop a single enterprise-wide database for their organization. Such visionary initiatives were anecdotally often reported to be expensive failures. 

• Some may remember the think-big waterfall styles of the 1980s, along with CASE (Computer Aided Software Engineering) tools? The backlash against Big Design Up Front included the proclamation of the Agile Manifesto in 2001, followed by a variety of think-small “agile” approaches. 

• Just a snippet from the world of data warehousing reveals Bill Inmon’s think-big emphasis on an enterprise-wide structure, followed by Ralph Kimball’s think-small approach with data marts, star schema, and more. 

• I’ll finish here — the 2000s saw think-big “centralized” data fabric, followed something like two decades later with think-small(er) “decentralized” data mesh.  

I concede that the above summary is incomplete, and many readers may challenge aspects of what I am saying. But here’s some key messages I hope we can at least somewhat agree on:  

• Think-big approaches often cost too much, and deliver too little, too late;  

• Think-small approaches often deliver tangible value in the short term but may come with technical debt, with the creation of data silos often being one common factor; 

And here’s the crunch. Over the years (and decades) good people with good intentions have sometimes swung to align with emerging think-big or the think-small approaches, but often either end of the spectrum comes with its own set of problems. So, we swing again. (More flip/flop?) 

For a moment, I’m going to ask you to assume that, at least in some settings, think-big approaches may have merit when they help think-small delivery to progressively assemble a unified view. So, it might at times have merit, but often think-big is pushed aside because it is seen as being cumbersome, or maybe even unachievable. Is there an alternative? Can we start with assembling a wholistic but light-weight framework in a timely fashion, then drive delivery of tangible value in think-small iterations?  

We turn back to town planning for some hints that might answer this question. 

Town Planning for a Brick and Mortar City 

The Story of Canberra 

Let’s start with a story that might make you smile. In the 1800s Australia was a collection of British colonies. In 1901, we became a nation (we could have called ourselves the USA – the United States of Australia – but unfortunately we’d been beaten to that acronym!) We needed a national capital city. Our two largest cities competed for bragging rights, but the peaceful solution was to build a brand-new city, Canberra, somewhere between the two.  

We could have dropped some good tradespeople with good tools in the open farmland and told them to just start building, but the smarter approach would be to develop a town plan first. We actually started building in 1911, started planning in 1913, and finished the planning in 1918 – seven years after we started building. Better late than never? 

We can laugh at that story, but let’s look at how some others towns might have been shaped. 

Three Types of Town Planning 

One extreme is what I call the anarchy style. Maybe some frontier towns emerged that way, with everybody doing as they please. That can happen in IT, too. Lots of IT projects are kicked off in glorious isolation. You get early traction, deliver value, but in the long run can end up struggling to manage an increasing number of silos. 

The other extreme I’m calling the dictatorial style. Nothing can start until the authorities have reviewed proposals in excruciating detail. We again can have parallels in IT, with various “authorities” seen as handbrakes, holding up progress.  

Now here’s another approach which I call the smart style, leveraging off a palette of proven, extendable patterns.  

For the construction world, in 1977 Christopher Alexander published patterns for small items such as windows and door handles, through patterns for a house, to patterns for cities and their regional surrounds. So how might this work for designing a city? 

A Bit More On “Smart” (Pattern-Based) Town Planning 

An Overview of Pattern-Based Smart Planning for cities 

Here’s an outline: 

1. Collect the patterns for universities, hospitals, shopping centres, and (lots) more. 
   
   
2. Engage the stakeholders to arrange these patterns in all sorts of ways as ideas for the city start to take shape. Importantly, people around the table will include highly qualified folk such as engineers, but also less technical representatives from the wider community. They can have fun, be creative, and if someone suggests placing the sewage treatment plant in the middle of a residential area, feedback can be given very early! From the many candidate solutions, one plan is chosen. 
   
   
3. Having agreed to one high-level town plan, the stakeholders can nominate what part of the whole is to be picked for this iteration’s actual delivery. The engineers and architects can then develop specific designs. It is important to note that the patterns that were understood by all at a conceptual level (universities, hospitals …) have lots of detail behind their façade to facilitate those who have to deliver bricks and mortar. 
   
   
4. Now let’s build! (Not the whole city, but just the bits nominated above, realizing that today’s delivery is part of a larger, cohesive whole). 
   
   
5. Iterate over steps 3 and 4 (and refine the earlier bits when new revelations warrant a revisit to steps 1 and 2). 

For context, please note that steps 1 and 2 are light-weight planning, while steps 3 and 4 are focused on delivery. 

An Overview of Pattern-Based Smart Planning For IT 

You probably were expecting this next bit, where we compare smart planning for cities with smart planning for IT solution delivery. With patterns, of course! We’ll use the same five steps: 

1. Collect the patterns, this time for products, customers, resources, tasks and more. Instead of looking to Christopher Alexander, we would do well to look at the published data model patterns from David Hay and Len Silverston (and my light-weight introductory patterns that often prove to be “sufficient” for data town planning – see my book reference for the latest version). 
   
   
2. Engage the stakeholders to arrange these patterns, with the stakeholders including “business” representatives (some senior management, some front-line workers), plus IT people. In collaboration, they can arrange the patterns to represent not only today’s data landscape, but reflect the vision for the future. Ideally, over weeks rather than months or years, the diverse views of a cross-section of the enterprise will be considered, and the views of all woven together respectfully to create the Data Town Plan.
   
    
3. The stakeholders can nominate what part of the whole is to be picked for this iteration’s actual delivery. The technical experts from within IT can then develop specific designs. As for Alexander’s patterns, the data model patterns often offer a wealth of ideas for technical implementation. 
   
   
4. Now, let’s build some IT solutions that will deliver tangible value in the short term but also fit in with the big picture. 
   
   
5. As we did for a city’s town planning, we iterate over the delivery steps 3 and 4, and where necessary, revisit the planning steps 1 and 2. 

The Next Article 

We’ve looked at some of the long-standing problems facing IT. Then, we took town planning for cities as a metaphor to gain some insight into alternative (good and bad) ways of managing our IT investments. The second (and final) article in this series involves the fun bit – looking at snippets from a “data town plan” for your enterprise, performing a reality check on the feasibility of producing massive value in a short timeframe, and a light-hearted story that warns of delivering data to the wrong “location.”