When Should Waterfall be Used Instead of Agile?

I just read a thread on the Scrum Development yahoo user group about when it would be appropriate to use agile and when waterfall would be better.

This brought to mind all the conversations I’ve had over the years regarding “when to use agile”. There are many folks in organizations that believe that waterfall is best suited for some projects.

So, I’ve been searching for those “types” of projects.

Now, let’s be clear. I am using the term “waterfall” in the strictest sense which means that:

  • Requirements have to be complete and signed off before technical design begins.
  • Technical design has to be complete before development begins.
  • All coding has to be complete for testing begins.

If we “kind of” follow this, then it is chaos.  The question we have to ask is … is it even possible to meet the strict requirements of waterfall?  My answer is NO for software projects, or any projects with a large amount of unknowns or required creativity and flexibility.

Can you ever successfully elicit and document a complete set of requirements at the most granular level?  This is what waterfall requires after all. If there is even a remote chance that a) the customer isn’t sure what they want, b) the customer may change their mind on what they want, or c) there is even a small amount of complexity, then it is very, very high risk and just not very smart to use waterfall. You just can’t…stop kidding yourself.

So, is there ever a reason why you would want to use waterfall? Sure! Here they are:

  • The culture is not ready to work cross-functionally.
    • Any agile methodology requires a cross-functional team. If the culture cannot sustain cross-functional work then you are forced into communication by documentation. Yeah, good luck with that. You should think about coming up with a strategy to make people hunger for cross-functional collaboration.
    • Not everyone “wants” to work in a cross-functional team environment. They need to either be coached up or coached out.
  • Management makes you use waterfall.
  • You can confidently define everything up front, there is little or no chance there will be changes, and there are no unknowns.

Let’s not pretend anymore that waterfall is actually appropriate for any complex software project with humans working on it.  With complexity come unknowns, and with unknowns, the amount of bureaucracy necessary to handle “change management” is ludicrous.  I have yet to experience any software development project that did not have at least some kind of complexity.  If you say “agile doesn’t work for all software projects”, then you have a misunderstanding of what agile truly is.

Up Front & Iterative Planning

You know the question…”what kind of projects are best suited for agile?”.  To me, this is the same as asking “what kind of projects are best suited for empowered teams, technical excellence, servant leadership, reducing waste?”.  Would there ever be a time when you would not want these things?  I know, that is a very smart-allec response, but I just can’t help myself.

I then ask “what’s the other option?”.  This surprisingly stumps people when I ask this.  We all know there is no such thing as “true” waterfall in software.  So, to help them along, I clarify what I “think” they are asking which is “what kinds of projects are best suited for big, upfront planning and design vs. iterative & incremental delivery?”.  People almost always agree with the restatement of the question.

I believe that everything can and must be done iteratively and incrementally in software.  However, the level of “up front” may change with the type of project.  Gutting out an old accounting system and replacing it with an Oracle or Great Plains solution will require more up-front planning and analysis than a green-field Web 2.0 social networking application.

So, fellow agilists, “up-front” is not a bad word, it has to be done.  We know this already with release planning, and even sprint planning, we just don’t call it “up-front”.  Just be careful…VERY careful.  You run the risk digressing into waterfall.  At times, there will need to be more “up-front” than other times.  Always be asking yourself, “what is the soonest we can deliver value to customer?”.

What makes a successful waterfall project?

I recently posted a question on linked in wanting to hear from people who have either led or been a part of a successful waterfall project.

Here’s the link. There a some awesome answers. What’s interesting is that what is required for a successful waterfall project is clearly defined scope, engaged sponsors, and a great, empowered team. Sadly, I rarely see all of these occur in the wild. And, these are the primary elements of what’s necessary for an agile project, explaining why there is a higher success rate with agile projects.