Agility versus waterfall in the age of the unique product

The key perceived strength of waterfall project management is the replicability of the process. However, very little attention is paid to whether the result is replicable. Here Alex Fishlock, CEO at DevOps consultancy Catapult CX, explains why agile is best when running software and service development projects.

Consider the example of a project initiation document in a waterfall project. The question of what content is in the document, and the quality of that content, is moot; providing the document is there, you can go ahead. The process is replicable, but the results could be anything.

However, from mass customisation in manufacturing to software development in telecoms, all of us are increasingly working on projects where the requirements for results are unusual. In DevOps and software development in particular, the agile mindset can deliver the most benefit for small teams, with loose governance, delivering high value projects.

So why do so many project managers run software development projects using the waterfall system? Especially when it is, frankly, much better suited for something like a traditional manufacturing line or physical retail installation.

For instance, in manufacturing, a project manager might run an improvement project on a line and then want to replicate that project on every other line in the factory. Meanwhile, in retail, a brilliant new fit out technique, tried and mastered in one store, might then be rolled out in thousands of stores globally.

In contrast, the agile mindset is beneficial in providing quick decisions and the ability to respond rapidly to changing conditions. It brings software to market and the extent to which its process is replicable is unimportant.

In most of the projects we implement on behalf of organisations developing software, we use Agile principles, and scrum, Kanban or even extreme programming. However, I often see people attempting to create a hybrid of waterfall and agile approaches, which normally results in a system that combines the weaknesses of both.

Simple, straightforward reporting

Reporting is good example of where a mixed approach to project management can come unstuck. A waterfall style project uses regular and formal reporting, at clearly defined steps. An agile approach involves regular stand ups in which each participant develops an understanding of the process, achievements to date and objectives. It would probably include a scrum board, often built in Jiva, or a Kanban board, to minimise bottlenecks.

One of the core problems with waterfall is that managers like to keep track of people and the time people spend on a project, but in agile the focus is on the problem and the value of the problem.

In agile, you might use storypoints as a unit of measurement to determine how much effort is required to complete a piece of work. This allows a team to focus on how much value the story is delivering, and how much value you are delivering in each sprint. It addresses the reality and importance of a result, rather than how many man days are required for the project and how much it ‘over ran’.

However, if you attempt to use an agile approach, with scrum boards, and produce time-based estimates expressed in a Gant chart, for instance, you mix methods, and cease to be agile, creating confusion.

Agile teams do report, but they report on the things that the board actually cares about – the effectiveness of the project and its deliverability. They use data from the project to improve the project itself, rather than justify the amount of time spent on a project. This removes the need for anyone outside the team to be concerned with the minutiae of the tasks and instead focusses on the delivery and effectiveness.

The danger is that you end up with a horrible and ineffective mix of the worst features of every methodology – a kind of #ScrummyAgileWaterBan that fails to deliver anything for anyone.

Instead, a better way would be to embrace Agile principles and engage senior leadership and management in stand-up meetings and scrum ceremonies, so they become involved and invested in the process.

Lighthouse teams and replicability

While replicability isn’t as important as delivery in a modern environment, where software is often unique to the organisation, it is important to be able to prove effectiveness. At Catapult, we use an upskilling system that we call the Lighthouse Model; whereby we identify a team from the ground-up that can act as a model for the rest of the business and focus first on developing them as a group.

By demonstrating the effectiveness of agile as a foundation on which to build software, a Lighthouse team creates a fertile environment, which removes blocks and gathers data to help develop buy-in across the board.

All this works. In 2018, the Standish Group established that ‘Agile projects’ are twice as likely to succeed than waterfall projects. In the same study the company notes that 28 per cent of Waterfall projects fail, while only eleven per cent of agile projects meet the same fate.

In this context, the metrics of success went beyond whether the project was on time and on budget and considered its outcomes and impact. They looked beyond the delivery against the plan to include the value delivered and customer satisfaction. In essence, they looked for the real meaning of success.

There should be no question that, in the current era of project management for unique software development, agile is king. #ScrummyAgileWaterBan meanwhile isn’t even a baron.

Accelerate software and service development in your company by learning more about Catapult CX’s agile product management here or read this case study about implementing agile on a massive scale.

About the Author

Alex Fishlock is the CEO of Catapult.cx and has over 25 years of experience in all aspects of software delivery and troubleshooting. Graduating with an MA in Electronic and Materials Engineering from the University of Oxford, Alex has taken two start-ups to IPO and held the role of CTO for a several public companies. Outside work he enjoys restoring old houses and collecting big data about everything in them. Catapult CX helps public and private sectors clients deliver better software and services. It provides customer-centric and agile product development, legacy technology modernisation, development of software-based services and is amongst the most longstanding Atlassian partners in the United Kingdom. The company holds ISO/IEC 27001:2013 and is Cyber Essentials Plus certified.

Featured image: ©ZinetroN