In the first article, I laid out the basic premise for this series: an examination of how Agile has gone from the darling of the application development community to a virtual pariah that nobody wants to be associated with, and an exploration of the very important question of what we should replace it with.
We started with a brief history of Agile, in an attempt to understand how it came about and what problems it was trying to solve. For years, I’ve been promulgating what I refer to as The First Law of Systems Analysis: In order to change anything, you first have to understand how it got that way.
In the second article, I enumerated a number of the criticisms that have been leveled against Agile, including many from my own writings.
In the third article, I tried to distill Agile down to a fundamental set of principles that can be applied to all projects, regardless of the type of project or the methodology being used.
In the fourth article, I examined the “lessons learned” from the Agile experiment, and noted takeaways that should be applied to all development projects.
In the fifth article, I discussed some of the consequences of misunderstanding or misapplying Agile principles.
In this article, we’ll explore possible alternative approaches to Agile that have been suggested as ways to streamline or otherwise improve the development process.
Alternatives to Agile
In the previous articles in this series, we have distilled Agile down to some basic, fundamental principles that should be applicable to all sorts of projects, regardless of the particular development methodology being used. Some of these principles include:
- Deliver value to customers as quickly and incrementally as possible.
- Ensure continuous interaction with the customer (even if the customer is not continuously involved with the project).
- Be transparent in communicating with all stakeholders, even if they’re not embedded with the project team.
- Accept continually changing requirements to the greatest extent feasible.
- Try to eliminate unnecessary and extraneous work.
- Support experimentation (within defined limits).
- Ensure freedom to fail (quickly and inexpensively).
- Create cycles of continuous improvement.
- Enable sustainable development, with time to rest, reflect, and refactor.
- Work positively and collaboratively.
- Automate processes to the greatest extent possible.
- Integrate testing with development.
- Empower people to organize themselves and solve problems.
- Provide supportive leadership throughout the project.
Now, let’s take a look at other approaches (other than traditional Agile) that also try to incorporate these principles:
1. Human-Centered Design (HCD) – The major assertion of human-centered design (aka design theory) is that projects are usually centered around products and product requirements when they should really be centered around users and user experiences. One example is the creation of the Great Western Railway in England by the great Victorian engineer Isambard Kingdom Brunel. When given the commission to design the Railway, he didn’t start by listing technical or engineering requirements, he began by envisioning how he wanted the user experience to be, from boarding the train at the station to disembarking from a steamship in New York.
When designing the Railway, he insisted upon the flattest possible gradient because he wanted passengers to have the sense of “floating across the countryside”. He constructed bridges, viaducts, cuttings, and tunnels all in the cause of creating not just efficient transportation but the best possible experience.[1]
Similarly, a project to create a new bicycle for adults started with an examination of the question of why 90% of adults don’t ride bicycles, and a project to design a new toothbrush for children started with an examination of several videos showing kids trying to brush their teeth with conventional toothbrushes. HCD teams are staffed with designers, behavioral scientists, anthropologists, marketers, and engineers, whereas most current project teams are staffed with business analysts and application developers.[2]
Human-centered design focuses heavily on stakeholder collaboration, brainstorming, prototyping, and iterative delivery. It also stresses a holistic, or systems, approach that includes not only the design and development of the product but also its marketing, adoption, use, and support.
Cliff Berg et al note that most Agile approaches fail to properly incorporate product design activities into their methodologies, leading to the creation of products with poor user experiences. They cite Apple as a counterexample of a company that emphasizes product design by having a design studio that’s separate from the company’s product development organization. One Agile 2 principle is that “Product design must be integrated with product implementation.”[3]
2. Six Sigma – Six Sigma is an approach that focuses on creating (and measuring) business value through the improvement of business processes. Approaches like Six Sigma focus on the business processes involved and explore ways of improving or eliminating them, whereas traditional development methodologies focus on products, This corresponds to the Agile 2 maxim that “the only proof of value is a business outcome.”[4] From a practical standpoint, the Six Sigma methodology has its drawbacks; for example, the approach is very rigid and inflexible, non-iterative, and tends to disregard the interests of legitimate stakeholders. Approaches such as Lean Agile (see below) have been developed to address some of the shortcomings of Six Sigma. But I like the emphasis on business processes and measurement of business value.
3. Lean Agile – Lean Agile (including methodologies such as SAFe) is an application of Six Sigma (or process improvement) practices to Agile development. Lean Agile emphasizes an understanding and improvement of the total value stream associated with a product, eliminating waste and unneeded work, emphasizing “just in time” and incremental delivery of product, and encouraging continuous improvement. One aspect of Lean Agile that’s emphasized is the creation of real-time feedback loops to inform the team of progress being made and problems encountered. Cliff Berg et al suggest that products have built-in feedback loops at three levels: the infrastructure level, the product engineering level, and the business outcome level.[5]
Another Agile 2 suggestion is to use a “dual track” system of development, in which a design team operates asynchronously from a development team, and each team feeds its output to the other team at designated intervals.[6] This can make for a more efficient workflow, especially on projects with a lot of design activities.
4. Kanban – Kanban is similar to Lean Agile in many respects but differs in others. It’s meant to be less disruptive to existing organizational practices than Lean or Six Sigma, allowing organizations to start from where they currently are. Kanban focuses more on managing work than managing people, and more on improving service delivery than on improving processes per se. Like Lean, Kanban makes use of feedback loops to identify issues, and emphasizes continuous improvement. Kanban also encourages self-organizing teams, and leadership at all levels of the organization.
Other, similar, approaches focus on what’s called “The Physics of Flow.” This is the idea that work should flow from one person or group to another in a “just in time” fashion, without any time-consuming “handoff” processes. Tasks to be performed are “pulled” by each team as needed, like parts on an assembly line, rather than “pushed” by outside directors.
5. DevOps – It may seem odd to mention DevOps in the context of Agile alternatives since in most companies, it is synonymous with CI/CD (Continuous Integration and Continuous Deployment). In other words, DevOps in most companies is just a Product Management technique. However, there is a larger sense in which DevOps is more than just a methodology, as expressed in the “Three Ways of DevOps” from author Gene Kim[7]:
The First Way – Systems thinking, described as an emphasis on the total value stream associated with a product or project.
The Second Way – Amplifying feedback loops to enable continuous improvement.
The Third Way – Creating a culture of continual exploration and learning.
As explained in Agile 2, DevOps, properly understood, incorporates much more than CI/CD; it encompasses Product Value (including both product design and functionality), Product Technical Design, Product Development Workflow, and Product Development Technical Practices.[8]
6. Domain-Driven Development – Domain-driven development (DDD) shifts the focus of both product development and product management from a central IT organization to individual business units. Each business unit determines what is needed to produce value based on its business needs, and creates and deploys products (including data products, applications, and services) locally. This approach has the advantage of being able to deliver value more quickly to the business. However, it also has drawbacks; for example, in many organizations, the central IT group still has to support, maintain, and troubleshoot these solutions (or portions of them), even though they weren’t involved in the process of creating them. Costs to the organization can skyrocket as each business unit “reinvents the wheel” to solve the same problem using a different approach (and different vendor products). And, as I pointed out in a series of articles on DDD, defining data and services at too granular a level can lead to both data quality issues and application/service performance issues.[9] Domain-driven development (including data mesh) can be made to work, but only if the proper amount of architecture and design is done at the organizational level first. It needs to be understood that data, applications, and services may have strategic value to an organization, as well as a tactical value to business units.
7. Software Engineering – Software engineering is generally what is meant when people refer, usually derisively, to the “Waterfall Method.” Agile approaches were created in response to perceived deficiencies in the software engineering approach. However, as I explain in my book “Building the Agile Database,” the two approaches are not completely incompatible.[10] Both, for example, emphasize product development, both try to reduce or eliminate “scrap and rework”, both emphasize attention to business needs and requirements, and both support iterative development and prototyping. Software engineering will often be the preferred approach for projects where the requirements are known and understood, where the application architecture is well-defined, where a high-quality, bug-free product is essential, and where the product will be supported and maintained by people not on the development team. A software engineering approach will be less applicable for projects where the exact solution to a problem is unknown, or where rapid delivery of a working product is more important than bugs or missing features, or where the application architecture is not defined, or where support for a product will not be handed off to another group. Also, as I point out in the book, a lot of the project overhead usually associated with the “Waterfall Method” exists for the purpose of satisfying the requirements of non-developer stakeholders (like managers who are trying to make sure projects come in on schedule and under budget). Software Engineering approaches tend to be more “Lean,” in that more emphasis is put on eliminating unneeded work and rework, while Agile approaches tend to accommodate more experimentation and “failing forward.”
So, which is the best approach? As I said earlier, it really depends on the requirements of each individual project. For each project, the approach chosen needs to keep in mind what is and isn’t known about the product requirements, what the immediate and long-term needs of the business are, what the requirements of each project stakeholder group are, what is needed to create the most value for the business in the shortest possible time, what the projects risks are (and how easily they can be mitigated), how clear the choices of solution architecture and technology are, how straightforward the design considerations are, and so on.
None of these approaches is guaranteed to work in all circumstances, or for all types of projects. All have their drawbacks, and each organization will have to weigh the benefits and drawbacks of any given approach to determine whether it will work well in their company culture. For example, several of these approaches (especially the Lean and Flow approaches) depend on the continuous availability of resources. As I pointed out in the second article, resources outside the development group (especially enterprise-facing resources) may be in short supply and not readily available when needed. Also, most Agile approaches are predicated on the idea of continuous improvement, incrementally growing an organization’s knowledge and capabilities over time. But too many companies rely on relatively inexpensive contract resources, downsize higher-paid “expert” employees, and pay little or no attention to the cultivation of knowledge and expertise within their organizations.
Organizations also need to consider the extent to which any of these approaches might challenge (or threaten) their corporate culture. Are companies willing to eschew micromanagement in favor of letting employees manage and organize their own work? Can they deal with the social, cultural, and behavioral issues inherent in Agile teams? Are they willing to abandon traditional “push” workflow practices in favor of a more Lean “pull” approach? Are they willing to provide the necessary levels of staffing? Are they willing to invest in new technologies (such as CI/CD) and new approaches (such as DevOps)? To what extent are they willing to cede centralized control of IT projects and decisions to business units? Jimmie Butler makes the point that all too often, development teams are in the position of trying to push Agile practices upwards in their organizations, because Management won’t accept Agile transformations at the organizational level.[11]
This series of articles isn’t intended to definitively answer the question of what development methodology to use, but rather to give you, the reader, a set of criteria to consider when choosing a development approach for any particular project. A certain amount of organizational trial-and-error will be inevitable, but over time you should become more aware of what approaches do and do not work for your organization.
1 Brown, Tim. Change By Design (Tim Brown, 2009), pp. 1-2.
2 For more information on Human-Centered Design and its applications, readers are referred to the IDEO website, IDEO.org. You can also download their Field Guide to Human Centered Design from this site.
3 Berg, Cliff et al. Agile 2 – The Next Iteration of Agile. John Wiley and Sons, 2021, p. 191-192.
4 Berg, Cliff et al, op. cit., p. 186.
5 Berg, Cliff et al, op. cit., p. 186.
6 Berg, Cliff et al, op. cit., pp. 202-203.
7 Kim, Gene. “The Three Ways: The Principles Underpinning DevOps”. ITRevolution blog post, August 22, 2012. itrevolution.com/articles/the-three-ways-principles-underpinning-devops.
8 Berg, Cliff et al, op. cit. pp. 181-182.
9 Burns, Larry. “Domain-Driven Development,” a series of articles for TDAN.com. Here is a link to the fourth article in the series; hyperlinks to the other three articles can be found in the opening paragraphs. htttdan.com/domain-driven-development-part-4/29883.
10 Burns, Larry. Building the Agile Database (Technics Publications, 2011), pp. 60-65.
11 Butler, Jimmie. “The Unintentional Intentional Distortion of Agile”. Blog post, no date. jimmiebutler.com/the-unintentional-intentional-distortion-of-agile/.
