A Business Analyst’s Preference: Agile or Waterfall?

Agile vs Waterfall Methodology

The question “What’s your preference?  Agile or Waterfall?”  is one of the more popular Business Analyst interview questions over recent years, and I was asked this exact question in the past.  This question comes with a catch.  Instead of understanding one’s preferences, what interviewers really want to know is whether this Business Analyst knows when, and under what circumstances, one needs to apply a particular methodology.

Deciphering between Agile and Waterfall Methodologies

Waterfall Methodology

Also known as a linear-sequential cycle model, the Waterfall methodology is one of the first process models to be introduced.  In a world where organizations have a clear idea on the end products and outputs, Business Analysts can work backwards to document requirements. Thorough planning is mandatory in a Waterfall project, with no overlaps in phases.  Requirements must be clear and unambiguous upfront, with thorough documentation throughout the entire project, more often than not including stringent review processes. Once this is complete, the requirements are utilized by architects and developers to start on the build, before this is implemented and tested. Once the tests are complete, the end product will be deployed and will be maintained thereafter. 

There are 6 phases in a Waterfall project:

1. Requirements: The potential requirements are methodically analysed and documented in a specification document that serves as a basis for all future development.  The result is typically a requirements document that defines what the solution should perform.
2. Design: This phase covers design elements, from stakeholder mapping to technical design requirements such as programming language, data layers etc. A design specification will typically be created that outlines business logic and what the product should look like, and details of achieving its end state. 
3. Implementation/ Coding:  Software coding is written in this stage, implementing all models, business logic and service integrations that were specified in the prior stages.
4. Testing/Verification: During this phase, quality assurance and testers work towards systematically discovering and reporting issues within the application that needs to be addressed.  When issues are reported, this will effectively move the project back into the previous phase until the issues are rectified.
5. Maintenance:  When reported issues are addressed, the application is deployed into the production environment and handed over to the operational users.  This phase consists of the subsequent support and maintenance that may be required to keep the solution functional and up to date. 

Although this was a heavily applied methodology pre-2000, organizations in fast moving and competitive environments often found the heavy ‘front-loading’ during the planning stages rather laborious.  This methodology aims to plan for the long term, which is less favorable to the many organizations operating under uncertain economic climates and having to re-plan from square one should parameters change.  The inherent lack of adaptability across all stages of the development life cycle may require dramatic leap backwards in a project, which may prove to be time consuming and costly for larger organizations.

The application of Waterfall is ideal when the end product’s requirements are fixed, yet time and money are variable. Although this was particularly applicable in heavily regulated industries such as electricity and banking, it is becoming increasingly difficult to pigeon hole entire industries since the larger organizations had started to embed customer feedback into their product offerings and business support functions which now needs to be managed in different manner.  Should such organizations choose a pure Waterfall approach in its entirety, it would be a case of too little, too late for product development, testing and deployment under an increasingly competitive environment. 

Agile Methodology

The emergence of increasingly sophisticated software applications had led to organisations experimenting with different ways of working towards faster delivery to market. First used in the early 1990’s, Agile is a popular methodology that took off in 2001 after the circulation of the Agile Manifesto and is an iterative approach to software development and project management. 

The core principles stated in the Agile Manifesto consists of:

• Individuals and interactions over processes and tools;
• Working software over comprehensive negotiation;
• Customer collaboration over contract negotiation;
• Responding to change over following a plan.

To deliver against these principles, projects are divided into sprints, which lasts between 2 to 4 weeks.  The focus of each sprint is to deliver or improve on a working feature, starting on the most critical functionality and working down towards to least crucial through prioritisation mechanisms.

Unlike the Waterfall methodology, Agile is not defined by a set of ceremonies and having teams with clearly defined roles and responsibilities.  Agile is primarily focuses on collaboration, customer focus, committing to feedback cycles and emphasising on cross-functioning teams to deliver to various sprints.  Project success is based on delivering value to the customers. Detailed documentation is consolidated into user stories, epics, technical notes and daily stand-ups, facilitating organisations to become more responsive in a competitive economic environment. 

Planning under the Agile methodology is iterative by nature and actions are flexible to changing circumstances.  A flatter project structure allows for regular collaboration between testers, developers and Business Analysts; many of whom may wear more than one hat/ play more than one role.  Requirements can be elicited and communicated to teams without the need for detailed documentation, and progress can be measured easily through delivery of working features.  

The pragmatism of Agile and its capability to adapt to a fast-changing world can be proven by statistics.  According to VersionOne’s State of Agile Report, 97% of organisations practice Agile in some form as of 2018.  However, it was noted that the adoption is not always widespread within entire organisations, signalling that there is still a way to go in terms of adoption and maturity. It could perhaps be the case that organisations are adopting Agile practices in specific business functions or projects, for example software development for new products and other methodologies for non-technical areas.

A number of studies had offered widespread support of the effectiveness of adopting Agile to their practices, with successes twice higher than Waterfall (Standish Group Studies, 2018), 25% saved on product delivery and 18% less defects after switching to Agile from Waterfall (Xplace, 2019).

The Rise of Hybrid Methodologies and what his means for Business Analysts

As the world becomes more uncertain and demanding, most organisations started to re-evaluate the value brought about by costly projects specifically aimed at delivering long-term benefits, preferring instead to focus on the new wealth of data that provided them with a much clearer sense of short to medium-term value.  As organisational strategies transform over time, most modern organisations who want to embrace a more Agile approach from their traditional Waterfall methodology often blend into a hybrid methodology, such as ‘Wagile’, ‘Structured Agile’, amongst other terminologies.  The rise of hybrid methodologies is often seen as the result of synergy gains from adaptive working.   

Whilst it is less complex to implement Agile into new start-up companies, the same cannot be said for large and complex organisations.  This transition may not come easily to some organisations if the personnel are adverse to changes and if this transition does not immediately meet business needs.  Therefore, adopting a mindset that focuses on agility is the critical success factor during this transition.  The agility comes in the form of cherry-picking the pragmatic elements from Agile to deliver value, such as increasing responsiveness to changes, individuals and interactions over processes and tools, selecting effective requirements gathering techniques over excessive documentation.  In short, the evolution of project methodologies and the rise of hybrid ways of working would not be possible without experimentation.  

It is therefore increasingly critical for Business Analysts to take a holistic view of their environment, gaining an appreciation of the rationale of the approaches used by project management and collaborate with their key stakeholders to determine the appropriate business analysis approach. Showcasing flexibility in approaches and tailoring business analysis techniques according to business needs will be ever more so embedded into Business Analysts’ competencies as methodology hybridization are fast becoming the norm.  

Conclusion – Agile vs Waterfall

Whilst all methodologies have their advantages and limitations, the critical success factor in any project as a Business Analyst in an ever changing world consists of firm grasps of both known and emerging project methodologies, application of such knowledge into one’s respective environment to ensure that the most appropriate business analysis techniques and stakeholder engagement mechanisms are being selected in order to deliver in a pragmatic and effective manner.