In the first article, I laid out the basic premise for this series: an examination of how Agile has gone from the darling of the application development community to a virtual pariah that nobody wants to be associated with, and an exploration of the very important question of what we should replace it with.
We started with a brief history of Agile, in an attempt to understand how it came about and what problems it was trying to solve. For years, I’ve been promulgating what I refer to as The First Law of Systems Analysis: In order to change anything, you first have to understand how it got that way.
In the second article, I enumerated a number of the criticisms that have been leveled against Agile, including many from my own writings.
In the third article, I tried to distill Agile down to a fundamental set of principles that can be applied to all projects, regardless of the type of project or the methodology being used.
In the fourth article, I examined the “Lessons Learned” from the Agile experiment, and noted takeaways that should be applied to all development projects.
In this article, I’d like to talk about the various consequences of misunderstanding or misapplying Agile principles and describe how these issues can be addressed.
Misapplications of Agile
I had intended to end this series with an article describing possible alternatives to the most commonly used Agile methodologies (that article will follow this one). However, I’d like to make a slight detour here and talk about the consequences of misunderstanding or misapplying Agile principles.
I was prompted down this path by an article[i] from Benn Stancil, describing the many feature and quality defects of software-based and AI-based consumer products, including the Humane AI pin and the Rabbit R1. Some of the defects include features promised, but not delivered, features that don’t work as promised, and features that work, but have obvious quality defects. The article includes the following quote from a YouTube product reviewer named Marques Brownlee (aka MKBHD):
A lot of these tech companies are developing tech kinda backwards. They’re delivering such unfinished products, that it actually makes them nearly impossible to review.
Like, it feels like it used to just be, make the thing, and then put it on sale. Now, it’s like, put it on sale, and then deliver the half-baked thing, and then iterate and make it better, and hopefully with enough updates, then it’s ready, and it’s what we promised way back when we first started selling it. And then this whole period in the middle is a mess.
Brownlee goes on to say it’s happening with other, more mature hardware products too. There’s “one major feature of every smartphone launch that gets announced, but that’s not coming until later in the year.” And cars, which now run tons of software as well, are getting delivered in “a half-finished state, where you just don’t get a lot of the features that you paid for.”
And generative AI products? Don’t get me started.
Benn Stancil makes the point that what distinguishes these half-baked products is the software they embody. Consumer products such as cameras, refrigerators, and even automobiles once had to work as advertised, or else the consumers could return the product for a refund, or sue for damages, or report the company to the State Attorney General or the Better Business Bureau. Now, consumers are being slowly conditioned to accept products that are shoddy, featureless, and defective, and they have to hope that, eventually, some software fix will be pushed out that will make the product worth what they paid for it.
This, to me, represents a major misapplication of Agile thought: the idea that it’s OK to rush a product to market (or to production) in a half-finished state, to be gradually and iteratively improved upon. In some cases, this may be OK, if (and only if) the consumers of the product understand and accept this condition, and if it’s more important to them to have a working product quickly than a bug-free one with all its features.
But this condition cannot be assumed; questions need to be asked and answered, and conditions negotiated and agreed upon. One of the problems with Agile, as I explained in the previous articles in this series, is that it’s based on an enormous number of assumptions, mostly having to do with the fact that the originators of Agile were all basically the same sort of people, doing the same sort of work. The signers of the Agile Manifesto were a group of white, male, middle-aged developers who met at a ski resort during a skiing vacation. This led them to make a number of assumptions around, for example, the efficacy of self-organizing teams and the lack of a need for leadership and management that would come back to haunt Agile methodologies later on. As Miriam Posner put it, “We’ve long known that eliminating bureaucracy, hierarchy, and documentation feels great, until you’re the person who needs rules for protection.”[ii]
I’ve also pointed out that the originators of Agile were not only the same sort of people, but they did mostly the same sort of work; that is, web-based and mobile software application development. These applications were mostly customer-facing, heavily centered around user interfaces, and it was relatively quick and inexpensive to create a minimally viable product (MVP) and incrementally add features to it. These applications were ideally suited to an Agile approach. However, many other types of software development projects aren’t. A lot depends, for example, on the importance of non-functional requirements (such as performance, scalability, security, privacy, quality, and maintainability) to the customer. We also have to take into account the requirements of other stakeholders (government safety and privacy regulators, for one), and whether application users need a fully functioning product out of the box or are satisfied with a less functional product if they can get it quickly. Again, these questions have to be asked and answered before a development methodology is chosen!
Alistair Cockburn, one of the original signers of the “Agile Manifesto,” has made the point that Agile was originally intended to support small, co-located development teams.[iii] Now, “Agile” techniques are being used to support massive global projects involving multiple and geographically dispersed teams, with predictably poor results.
And finally, the originators of Agile were products of the same Silicon Valley culture. One member of the Agile Alliance described this culture as centered around “innovation, freedom, entrepreneurship, collaboration, shared ownership, and anarchism.” But these values are more suited to some types of people (extroverts, for example) than to others. I’ve already noted the extreme anti-management bias of the Agile Alliance, arising out of what I sometimes refer to as “The Dilbert Culture.”
Lindsay McGregor and Neel Doshi[iv] have noted that organizations, in their attempt to make Agile mean all things to all projects, have created processes that are directly counter to the original principles of the “Agile Manifesto”:
- Individuals and interactions over processes and tools — But in most organizations, “Agile” means implementing a framework process such as SCRUM, SAFEe, or Kanban, and supporting that process with tools. No one is allowed to question the process, or work around the tools.
- Working software over comprehensive documentation — But how much time is spent on “Agile” projects creating and editing the user stories and product backlog, test documents, the burndown charts, etc.?
- Customer collaboration over contract negotiation — Yet, as I’ve pointed out, most of the effort on “Agile” projects is spent defining, developing, and delivering products, rather than understanding a company’s strategic objectives and value proposition, and working with the business to create value and improve processes.
- Responding to change over following a plan — But when project teams are not focused on a company’s strategic objectives and value proposition, most “Agile” projects merely become exercises in following the prescribed framework: holding endless meetings and ceremonies, producing endless project artifacts, etc.
In the second article in this series, I’ve listed a number of criticisms of Agile, and I won’t bother to reproduce them all here.[v] But a few deserve special mention, including ignoring non-developer Stakeholders, focusing on the creation of products over the improvement of processes, ignoring front-line business value, and not taking a holistic (i.e., systems) approach to the problem being solved.
This last one is particularly important in view of Stancil and Brownlee’s critiques of the tech industry (see above). Taking a systems approach basically means making sure you’re doing the right thing (or solving the right problem), as opposed to just “doing the thing right” (i.e., following a correct process), which is mostly what Agile methodologies are concerned with. I’ve mentioned before that Agile methodologies (actually, all software development methodologies) focus on product delivery to the exclusion of everything else, including ethical concerns.[vi] Not only are the tech companies delivering incomplete, buggy, and half-baked products, they seem to be delivering products that no one really wants or needs, and then expecting a market to magically create itself. Who, for example, really wants or needs Amazon’s $3,500 virtual reality glasses? Why are we spending billions of dollars trying to teach AI programs how to look up information from databases? Do we really need self-driving cars (especially since they don’t seem to work)?
At the risk of sounding repetitive, I am going to assert that no methodology of any sort is going to be successful, unless it does the following:
- Embraces Stakeholder Concerns — The legitimate interests of all affected Stakeholders (including, but not limited to, customers, business partners, product users, employees and regulatory agencies) must be thoughtfully understood and taken into account.
- Takes a Holistic View — The problem to be solved must be understood in its widest possible sense, including the environment in which the problem exists and its effects on Stakeholder groups. We need to be sure we are solving the right problem and doing the right thing.
- Improves Processes — The purpose of a project is not just to create a product (like a software application); in fact, it can be argued that the creation of the product is the least important aspect of a project! What is more important is to ensure that a value-generating process is being created or improved by the project.
- Optimizes Work — As mentioned before, Agile processes aim to maximize the amount of (unneeded) work not done, and to get the maximum amount of value out of the work that is done.
- Creates Value — We need to shift our focus from the creation and delivery of products to the creation and delivery of front-line business value. Our business customers don’t care about hardware or software or development methodologies; they care about the creation of business value.
- Increases Organizational Capabilities — The completion of each project should result in an increase in an organization’s knowledge, understanding, expertise, and ability to solve problems and deliver business value.
In the next (and last) article in this series, we’ll examine some possible alternatives to current Agile approaches, and discuss the benefits and limitations of each.
[i] Stancil, Benn. “The SaaS Counterculture”. Substack blog post, May 3, 2024. benn.substack.com/p/the-saas-counterculture.
[ii] Posner, Miriam. “Agile and the Long Crisis of Software”. Logic Magazine, March 27, 2022. logicmag.io/clouds/agile-and-the-long-crisis-of-software/.
[iii] Cockburn, Alistair. “I Come to Bury Agile, Not to Praise It: Effective Software Development in the 21st Century”. Agile 2009 conference presentation. infoq.com/presentations/cockburn-bury-not-praise-agile.
[iv] McGregor, Lindsay and Neel Doshi. “Why Agile Goes Awry, and How to Fix It”. Harvard Business Review, October 1, 2018. hbr.org/2018/10/why-agile-goes-awry-and-how-to-fix-it.
[v] Burns, Larry. “The End of Agile – Part 2 (Critiques of Agile)”. TDAN.com, April 3, 2024. tdan.com/the-end-of-agile-part-2-critiques-of-agile/31699.
[vi] Why, for example, are we rushing to monetize generative AI before resolving the issues of copyright infringement and theft of intellectual property? I’ve mentioned before that the whole “Move Fast and Break Things” tech bro ethos only works if two conditions are met: First, that the cost of “things breaking” is minimal, and second, that this cost is not borne by critical stakeholders. Moving Fast and Breaking Things doesn’t work well when you are the thing that is being broken.
