Something went wrong with Scrum: Analogies with the Underpant Gnomes and Sovjet Era Deathmarches

Story card hell is when you have 300 story cards and you have to keep track of them all. [...]
Every customer I've ever worked with wanted to put story cards in a database. That misses the point of story cards. If you're worried about losing story cards, you're doing too much with them. 

- Jim Shore, Beyond Story Cards (2005)

 

When I was getting to know Scrum, I was taught that it was very much about reacting to change. Product owners define and prioritise the product backlog. They can react to change by re-prioritising the product backlog. Sprints are short focused iterations where the team is protected from change and the associated task switching costs. The effort is streamlined during this period. But because Sprints are short, reacting to change is still possible on the level of the project.

Scrum poses several challenges for the product owner:

• He is challenged to break down backlog items so that they fit into sprints and are still valuable and independently deliverable.
• He is responsible for the detail-level of the items in the product backlog. This is a balance-act since the items on the top must be detailed enough for the team to implement them. However items lower in the backlog should not be too detailed, since they can change completely or even become obsolete during the course of the project. Spending too much time upfront on those items is wasteful. I was taught  that items that are not on the top of the product-backlog should just be placeholders for a "just-in-time specification".

It seems that the current state how Scrum is practiced does not reflect those ideas any more. The following burn up-diagram from a real project I have been working on is a good illustration:

The large gap between "Scope" and "Work Done" hints at a chain of problems:

• The product-backlog is used to plan the whole project and to monitor its progress.
• In order to plan and monitor the product-backlog should be as exact as possible. Therefore all items on the product backlog must be estimated as exactly as possible.
• When items on the backlog are estimated, they are no longer "placeholders for future refinements". They become specifications.
• An exactly estimated product-backlog suggest a certainty we simply don't have.
• Following from the above, the backlog strives to become a complete specification for the whole project.
• Reacting to change is no longer a main concern, when this complete specification is in place.
• … where is now the difference to a Waterfall Project?

 

It seems that my experience is not unique:

 

In his talk "Co-making Great Products" Jeff Patton draws a hilarious analogy between collecting a huge Scrum backlogs and the business plan of the Underpants Gnomes in South Park:

Gojko Adzic is even a bit more provocative (as always). In his talk "Make Impacts, Not Software" he compares the "Complete-Backlog-Antipattern" in Scrum to the Sovjet Era deathmarches like the construction of the White Sea Canal which was a showcase of the success of the First Five-Year Plan:

 

 

According to a recent Forrester Study the above way to practice Scrum is the mainstream today. Forrester calls this "Water-Scrum-Fall":

The reality of agile adoption has diverged from the original ideas described in the Agile Manifesto, with many adoptions resembling what Forrester labels water-Scrum-fall.

 

According to that study Scrum is not adopted by the whole company. Scrum is seen as the "Development Process". The input for that process is a long term roadmap or a complete feature list defined by the business. Then the development team does all their esoteric ceremonies like "Standups", "Sprints" and "Demos" but when you wait long enough, the output will finally be the complete delivery that goes to production.

 

I personally don't think that we gain many benefits from "Water-Scrum-Fall". In some cases it can even be harmful, and the project would be better off with a honest waterfall process. Gojko makes the analogy of putting a strong engine into a crappy car.

 

However I think that this is an inherent conflict between Agile methodologies and plan driven approaches. If you really need a plan driven approach (i.e. because you have a defined long term roadmap) then probably all attempts at using Agile methodologies are just pretenses and will eventually fall back to a disguised waterfall approach.