Not happy with Agile, but why?

We were used to getting shit done … then they told us about Agile.

- Hadi Hariri, NDC 2012

 

I stated that something went wrong with Agile. This probably boils down to how Agile is applied in the Enterprise. And I am not the only one stating this.

But what went wrong and why? Criticising and pointing out problems is easy. But finding the root cause for being unhappy requires more reflection.

So what does Agile really mean?

We all know the Manifesto for Agile Software Development. But that are some high-level principles that were formulated more than a decade ago.

Software development looked radically different at that time. In the meantime we have gathered about a decade of experience in Agile Software Development and the software development landscape has changed quite a lot. If you believe some studies then Agile is now mainstream and the most widely applied methodology in software development.

Of course the values from the Agile Manifesto are still valid today. They still are the base for an agile mindset. I guess they are commonly accepted today in the industry and hardly anybody will directly disagree with them.

But it became clear that the Agile Manifesto is not enough. Even if we agree with it in principle, do we reflect it in our every-day actions?

Some people tried to go further with more manifestos:

• The Manifesto for Software Craftmanship
• The Software Team Leader Manifesto

However it became obvious that the manifestos and the reality in the trenches of the enterprise were not congruent:

• The Manifesto of "Programming, Motherfucker"
• Manifesto for Half-Arsed Agile Software Development

Today we have at least 12 Agile Methods. But are they any relevant for us as developers?

A lot of times when I speak with other developers, then they say that "Agile" is actually just common sense. A lot of developers claim to have been "doing stuff the agile way" before it was even called "Agile".

So, what does it mean to "do stuff the agile way"?

My favourite interpretation is what Jason Yip calls "Agile as a Doctrine" in his post: What do you mean when you say "Agile"?:

1. Reduce the distance between problems and problem-solvers
2. Validate every step
3. Take smaller steps
4. Clean up as you go

That is to the point what we developers do when we do Test-Driven-Development. It even does not have to be "pure" TDD: It's the Mindset not the Tool! 
It's also reflected in the basic methods engineers or scientists learn to practice (build up knowledge, take small steps, validate expectations).

Further more it's what most of us practice in some form to manage our personal live. Implicitly with "common sense" or more explicitly with GTD or Personal Kanban: we know that we have a limited amount of time and we (more or less successfully) optimise to get the most value out of it. We do not optimise for productivity, we optimise for value! We can do that, because its our lives.

If we as developers already do "stuff the agile way", where is then the problem in the current state of software development?

Here it comes: The software development industry today is not about the craft of writing software, it is mostly about management! At least in the environments where most of the software developers work.

And that is where Agile is currently failing. Despite the fact that the principle of Plan-Do-Check-Act is taught in most basic management courses, in the enterprise environments I have been involved, things are not done the "agile way":

• Instead of reducing the distance between problems and problem-solvers we introduce more layers of indirection with several layers of "architects", "product owners" and "customer proxies". We add several layers of processes for roadmaps, plannings and priorizations. We introduce portfolio-, product- and release-management and build up several layers of backlogs. (example)
• Instead of validating small steps we focus on long-term roadmaps. We increase our effort in crafting complete backlogs. We believe that tracking activities on a micro level will somehow lead us to success.
• We have a hard time focusing on value, therefore we keep ourselves busy with optimising for productivity.

For developers this environment is especially frustrating, since "doing stuff the agile way" works on the technical level and also on the personal level. It just doesn't work on the management level in typical enterprise environments.

Lately there are fresh movements that try to address the chasm between Agile development and Agile management:

• Impact Mapping
• The Lean Startup Movement
• The NoEstimates Movement

Those movements try to address shortcomings in the current adoption of Agile in the Enterprise. 

But I remain sceptical. Of course we can still go a lot further in adopting Agile on all levels. But I have the feeling that Agile just does not scale well on big projects or on big organisations. The bigger a project/organisation the bigger the need for structure and formalism. And structure and formalism prevents "doing stuff the agile way".

So how can we developers deal with that?

I think as developers we have almost no chance to influence the Agile adoption in the typical enterprise environment: Since it has to happen on management level, once you would be in a position to really have an influence, you are not a developer any more.
But as developers we can choose who we work for. This is especially true with the current shortage of developers in the industry. I am confident that there are environments out there where software development can happen in a more agile way on all levels. I even think that because of current trends in the industry these opportunities will increase in the future. More and more small organisations prove that there is less and less need for big corporations in the IT industry. 

Of course big corporations have their advantages. Security and stability are often among them. So every developer has to decide for himself if "doing stuff the agile way" is really worth it in comparison.

Finally, how do we developers find those jobs? I think it's by really looking at the management of a company. What does management really do? What are the management processes?
Github Inc. is the current poster-child of an Agile company. In a talk about their model of distributed management they summarise my above rantings with this venn diagram: