Agile and Iterative Contrasted with Waterfall
We rely on short quick development steps, feedback, and adaptation to clarify the requirements and design. To contrast, waterfall values promoted big up-front speculative requirements and design steps before programming. Consistently, success/failure studies show that the waterfall is strongly associated with the highest failure rates for software projects and was historically promoted due to belief or hearsay rather than statistically significant evidence. Research demonstrates that
iterative methods are associated with higher success and productivity rates and lower defect levels.
I especially like his description of the up-front requirements and designs as "speculative." In the interest of keeping a project on schedule, the project manager following a waterfall approach will press the user group and business sponsor to sign off on a set of requirements and designs so developers can begin coding. Once the requirements and designs have the sponsor's sign off, all changes are managed through change orders with cost implications. This has the negative effect of penalizing the sponsor and treating the business users as if they don't really understand their business processes very well. In reality, the business users understand the de facto process very well, but usually are not adept at describing it at the level of detail that a final set of requirements demands. They are more likely to "know it when they see it." That is, when the business users can see prototypes or the first pieces of working code delivered and can put their hands on the keyboard to begin using it, then they can speak more authoritatively about what works and doesn't work for them in support of their business processes.
An agile, iterative develpment approach allows all stakeholders to act with greater honesty and integrity about the software development process and therefore has a greater chance of succeeding.
For an more thoughts along these lines, see The Challenge of Requirements by Abhijit Nadgouda and Software Requirements are Elusive: 6 Reasons Why Customers Cannot Get Them Right.