Please welcome Charles Betz and his new column, The Digital Transformer, to the pages of TDAN.com. Charlie has contributed materials to TDAN.com in the past and we look forward to sharing his words of wisdom each quarter.
Is Agile Killing Enterprise Architecture?
The relationship of Enterprise Architecture (EA) to the Agile movement has been a topic since the earliest days of Agile, with Scott Ambler being a notable pioneer. More recently, Jason Bloomberg has questioned “Is Enterprise Architecture Completely Broken?” and discussed Agile EA at Netflix with the notable Adrian Cockcroft.
Various authors advocate that practices like iteration, Scrum, incremental delivery, and DevOps be combined with enterprise architecture, but I think that some of these discussions are superficial. In light of the current challenges, we need to have a deeper understanding of what EA does and how it adds value. Yes, you CAN weave EA into an Agile delivery stream. But why SHOULD you?
This column series (probably 2 or 3) will discuss EA objectives in terms of:
- Lean product development
- Human visual processing
- The concept of the OODA (Observe/Orient/Decide/Act) loop
- Lean objectives on waste reduction
I’m assuming that EA is concerned with coordination across broad portfolios, and reducing redundancy in acquired and developed products — in other words, EA as zoning authority, not solutions design. I use the term “product” generically to include IT service, application, or acquired software, whether internal or external-facing.
Lean product development
The work of Don Reinertsen (Principles of Product Development Flow) and Eric Ries (Lean Startup) is rich in insight for EA. Reinertsen views product development as the production of information, a powerful insight. Both he and Ries state that we generate information through a quasi-scientific process of hypothesis and falsification. Reinertsen further emphasizes the need for an economic framework to guide all decisions, one especially attentive to Cost of Delay. EA must support these principles.
An EA future state diagram is nothing more than a hypothesis, and EA needs to be alert and adaptive to its falsification. Waterfall and command-and-control EA is inconsistent with this principle and therefore is harmful.
However, the way information is generated in product development is through the “pruning” of unprofitable avenues – falsification and “failing fast.” (The term “fail fast” has become an overhyped meme of late; Reinertsen in Chapter 4, “Exploiting Variability,” discusses the economics and statistical theory behind the hype.)
EA can play an important role here, as architects are tasked with understanding broad dynamics and cross-system interactions, and also may have significant domain expertise in various classes of technology. EA also seeks to reduce variability and waste in the operating environment. These concerns are grounded in Lean, and to the extent that the EA group is effectively performing them, they are adding value.
In order to deliver value in this way, the EA team needs to match the cadence of the Agile teams. This is a central challenge.
There are many other EA insights to be drawn from these authors. They will be explored in future columns.
Visualization
EA draws pictures. Architects in general don’t write code; instead the most visible aspect of their work takes the form of diagrams — abstract graphical representations of complex systems. This has been problematic in the Agile community as some IT professionals, including perhaps some of the most skilled software engineers, find little use in diagrams. (See Martin Fowler’s Is Design Dead?)
It is commonly thought that everyone needs to maintain some technical competency, typically through writing code. Sriram Narayan in his excellent Agile IT Organization Design: For Digital Transformation and Continuous Delivery calls for designing “an organization where planners [e.g. architects] are required to spend, say, 20% of their time in execution” which he suggests would consist of “writing code or performance tests or simply trying to address technical debt in some area of the codebase.”
However, Dan Moody, in his seminal paper The Physics of Notations, observes that:
Visual representations are effective because they tap into the capabilities of the powerful and highly parallel human visual system. We like receiving information in visual form and can process it very efficiently: around a quarter of our brains are devoted to vision, more than all our other senses combined … diagrams can convey information more concisely and precisely than ordinary language [8, 68]. Information represented visually is also more likely to be remembered due to the picture superiority effect…
…Visual representations are also processed differently: according to dual channel theory [80], the human mind has separate systems for processing pictorial and verbal material. Visual representations are processed in parallel by the visual system, while textual representations are processed serially by the auditory system…
Because diagrams are more readily processed, they are often used to represent high level system interactions – how a given service, product or application is related to peer systems and services. Building such depictions can be helpful to fostering a shared mental model of the overall system objectives and context, and crucial to orienting teams in complex environments. EA can add value in making such visualizations easily available and re-usable, and clarifying the shared mental model they represent (e.g. through injecting metamodel rigor). Should architects prioritize writing code over visual work? I think it’s a question with an open answer.
OODA loop
Humans go through a defined process in building their mental model of complex and dynamic situations. This has been formalized in the concept of the OODA loop, popular of late in the Agile community. Standing for:
- Observe
- Orient
- Decide
- Act
The OODA loop is a concept deriving from US Air Force doctrine, in particular the work of John Boyd, a man who broke down the mental processes of Korean War fighter pilots into the OODA cycle. He and others have extended this concept into various domains, including business strategy.
Especially in complex enterprise environments, it is essential for people to quickly ground their individual observations in a broader shared mental model. A well represented and easily accessible enterprise architecture should help people understand the purpose and relationships of any arbitrary IT resource in their environment. Therefore a conjecture: Enterprise Architecture provides value in the Orienting phase of OODA.
Conclusion
I think there is much to guide EA in applying these ideas, but I also think there is much to support the continued existence of EA as a coordination and governance function. If we understand EA in terms of its underlying motivations, we can better create truly agile EA.
Note: I am working on a longer paper on these ideas – if you are interested seeing it please drop me a line.
Next column: EA as waste reducer.
