Agile Testing

Agile Testing Myths

by
Almost every existing process comes with it’s own share of understanding at individual level. Software Testing itself has always been a debatable topic. It is surrounded by many myths all of which have counter facts associated with them. Head on to the post – Software Testing Myths and Factsto read some of such misconceptions. Therefore, Agile Testing Myths are there out in open aswell.
Agile Testing Myths
Coming back to Agile Testing, below are common assumptions many teams have:

•Agile means

•ZERO documentation
Documentation is a critical process project teams run away from. With Agile in force, it is a generic response to get that – “Since we are following Agile, there is no documentation of requirements and other details”. Actually, If followed sincerely, The end result of an Agile project would result in much better and detailed documentation which would be well structured, easy to read and follow.
•No planning
Ask an agile team for their project plan, the maximum they would be able to provide is the current or current + next sprint backlog. On the contrary, Agile process insist on having the sprint backlogs ready for current along with upcoming 2 sprints along with all the estimates and acceptance criteria. Thus, in practice, planning in agile projects is more rigorous.

•Unit testing

•Is sufficient and thus remove the need for manual testing
It is a common misconception that if Unit Testing is performed effectively, testing activities may be skipped. While it is true that unit tests play a significant role in reducing bugs, the fact that they cater to specific and limited scope must be considered as well.
•Can be built into regression suite
In continuation to the point above, creating a regression suite from unit tests is also seen as a practice in some teams. Projects must understand that Regression Testing is altogether a separate activity with it’s own scope. While some of the unit tests would always be common to both regular test cases as well as Regression Test Cases, compiling all the unit tests in one place and calling it a regression suite is a complete no-no!

•Automation

•Is impossible
Test Automation in Agile development projects is sometimes considered an impossible task! The main reason for the same is lack of time. But then, I think, the remedy to this points to the very first myth – “Agile means No Planning”. The need of Test Automation and scope for the same must be a part of planning process from the very beginning.
•Remove the need for manual tests
With automation in place, projects tend to avoid manual tests in respect of time and resources. Since, It is not possible to cover each and every case in automation, Manual Testing can never be replaced by Automation Testing. Even if the tests show 100% coverage in reports, exploratory testing generally done unintentionally as a part of manual testing would be at loss.

•Testing is not a part of ‘Done’

It is quite typical with the development teams to mark a story done after implementation from their end even before it is tested. Sometimes, to get the burndown chart right, teams even move the user story to “Done” state if it has a bug associated with it. The explanation given is that the bug thus raised is already in backlog, so the user story should not move forward.
In ideal practice, any story is not “Done” until is has been tested successfully with no pending bugs associated with it.
More Topics in this series:
Testing in Agile
Approach to Agile in Testing
Challenges for Testing in Agile
Advantages of Testing in Agile
Principles of Agile Testing
Best Practices in Agile Testing
Agile Testing Myths

Best Practices in Agile Testing

by
In the previous posts, I covered basics of Agile Testing. Those include It’s definition, approach, challenges, Advantages and Principles of Testing in Agile. Here, is another very basic but important topic which is generally underrated – The Best Practices to follow while Testing in Agile environment.

•       Unit test is a must

Unit Test – Smallest piece of code that can be tested individually.
This smallest piece of code is the one which is ignored most commonly. Test execution activities begin with Unit Testing and thus form the foundation of quality checks. Ironically, unit testing is the one which is most commonly ignored or cut short upon.
The teams must understand the importance of Unit Tests and make sure they are being executed. Automation is also another important factor which broadly help unit testing.

•       Minimize dependencies

One of the most important attribute of an effective test case is that it can be executed by anyone in the team irrespective of who authored. The same principle applies to Agile too. The tests must be written in a manner that they have minimum dependency on test data and other configurations and thus can be executed and used to generate reports by anyone, anytime

•       Follow Test Driven Development (TDD) or Behavior Driven Development (BDD) appropriately

TDD and BDD are the latest buzz words when it comes to boiling down another buzz word – Agile! While these techniques are definitely useful and prove very handy, care must be taken while implementing the same. It is not always necessary that the models fit into

•       Automate

     Test Automation is yet another field which is always in discussions and looks really attractive. It’s a different topic of discussion, but for now, below are the areas which can be good candidate for for automation:
•       Unit Tests
•       Regression Tests
•       Deployment

•       Create Scenario/Story-based Test Cases

      With the development being done in Agile and releases are done as a collection of user stories,  testing activities should not be ignored and follow similar procedures. It’s important to write test cases based on individual user stories which are further linked appropriately to the Epics.
Topics in this series:

Advantages of Testing in Agile

by
With Agile being the latest hashtag, more and more organizations and projects are taking steps to adapt the agile software development process. At the same time, most of the teams which are going agile or planning to move into the direction of Agile process are still reluctant to include Software Testing on the same page. Benefits of testing to projects are now widely known and all of them apply to Agile projects also. Below are some major points which can be re-emphasized when talking about the advantages of Testing in Agile.
•Improved Quality
Quality is a wide term and without any doubt the responsibility of whole team which comprises not only testers, but also all other stakeholders. When testing is involved right from the beginning of project, provide valuable inputs to taken required actions to improve the quality of products.
•Saves Resources
Budget – both in terms of time and money is the prime concern of clients and they consider it as main resource.
Early detection of bugs has been the mantra to minimize the effort of fixing the same. Needless to say, that when the test teams are involved continuously, helps the teams to progress in this direction. When there are less bugs in the applications or the products, clients see a drastic improvement in the utilization of their resources – time and money.
•Improved Communication
Effective communication is the key to the success of any Agile project. Making testing as an integral part of the Agile Framework ensures that the team is aware of all(or at least most of them) possible scenarios and thus helps them achieve the target.
Since testers have a different mindset than developers, the whole team can thus work together and collaborate better. Also, the clients then adapt a better picture of the product and understand the details of functionalities rather than being caught in the scenario of coming back with a production issue when they face it.
These are the major Advantages of testing in Agile that I have come across. Do share the ones you think of too in comments.

Challenges in Testing in Agile

by
‘Software Testing’ itself has always been going through many challenges. After decades, the concept was now clear and teams have realized the importance of same. Now with the Agile approach being the buzz word, teams are already struggling to keep the procedures, templates and quality in sync. Agile based projects generally go through a lot of challenges by themselves and software testing is often neglected as a result. Below are the major challenges testing teams face while working in Agile teams:
•       Changing Requirements
Frequent changes in requirements is the first thing which comes into picture when we talk about Agile environment. This feature is the first and major challenge QA teams face in projects facing Agile methodology. Since the requirements keep on changing, it becomes bit difficult to regularly update the test cases and ensure test coverage.
This can be overcome if the testers and developers are on the same page about progress modifications made in each other’s activities.
•       User stories with no description
One of the myths around Agile methodology is ‘zero documentation’. It is this misunderstanding which leads to one liner user stories or stories with too less information.
This challenge again need inputs from both development and QA team. Developers need to understand that while Agile methodology discourages bog documentation around every detail, basic documentation of the functionalities is still mandatory and required. At the same time, QA team also need to accept the user stories with less information and try to take as much as they can on a high level from them. Such test cases can always be marked with a filter and updated as and when the information is available.
•       Waterfall inside Agile
This is another major issue in many projects when the sprints become available to the QA team only towards the end of it. The testers then struggle hard in order to maintain test cases and performing other activities in a short period of time.
Making sure that the User Stories are available to all stake holders right from the time of their creation is very important for a successful Agile implementation.
•       No time for Regression
Availability  of User stories to the test team close of sprint end leaves little time to execute regression test cases. The importance of regression testing is something everyone agrees these days. Thus, incomplete regression testing is one of the reasons of UAT defects.
Timely delivery of user stories is one way this can be rectified. Automation also plays major role in overcoming this challenge. Test Automation for unit tests and regression tests may significantly help.
•       Communication gaps
If we look back, most of the challenges arise out of one or the other form of miss-communication. Merely formulating great processes and adopting latest methodologies do not guarantee the success of any project. The project is executed with high quality when all the teams and stake holders work together and help each other in every phase.

Topics in this series:

Testing in Agile
Approach to Agile in Testing
Challenges for Testing in Agile
Advantages of Testing in Agile
Principles of Agile Testing
Best Practices in Agile Testing
Agile Testing Myths

Approach to Agile in Testing

by
In my previous post, I talked in brief about “Testing in Agile“. Following the same, here are the approaches that may be followed for testing activities in Agile Framework.
 
Following the right approach to testing is a crucial factor to incorporate successful testing activities in Agile process. Since the development model is different than traditional methods, the teams need to sit together and finalize a process that best suits their project. Generally, following two approaches may be applied:
•       Testing in Same Sprint
This method is like the traditional testing process where the QA team perform their test cases in the same sprint/Module (as per traditional process). Ideally, this should be the right approach to follow. Here, the QA team works on the same user stories as development team and prepare their test cases, perform execution iterations and make sure that the sprint delivers a quality product.
•       Sprint lag – Sprint+1: Separate QA and DEV sprints
Since testing itself in Agile has its own challenges and tight deadlines is one of the main risk, most of the times, QA team is left with insufficient time to complete their execution iterations. This may happen due to continuous changing requirements which lead to last minute coding in development phase, deployment delays and many more factors.
To overcome this problem, some teams follow the ‘Sprint lag’ or ‘Sprint+1’ method. In this method, development and QA teams work on different sprints. For example,
Sprint 0:
Development Team – Works on Sprint 0
QA team – Works on test case preparation of Sprint 0
Sprint 1:
Development Team works on
1.    Sprint 1 development items
2.    Sprint 0 bug fixes
QA team works on –
1.    test execution iterations of Sprint 0
2.    test case preparation of Spring 1
Sprint 2:
Development Team works on
3.    Sprint 2 development items
4.    Sprint 1 bug fixes
QA team works on –
3.    test execution iterations of Sprint 1
4.    test case preparation of Spring 2

Testing in Agile

by

Anyone in the world of software development, is not new to ‘software testing’ in Agile environment. It basically means performing the various testing activities for applications or software or products which are developed under the Agile model.

In this post, I have tried to touch the topic on very broad scope. We will just go through the very literal meaning here.

 What is Testing?

• The process of evaluating a software application/project to detect and report differences between given input and expected output in order to make sure that it meets the customer requirements.
•An activity or process which involves verification, validation of a software program to find out if it meets the business and technical specifications of requirements and providing a detailed defect report at the end of every iteration.

What is Agile?

Agile is the latest buzz word. Everyone is either adapting Agile Methodologies or want to go in the same direction. When we hear, or say Agile, following points come into our mind:
•-Ability to move quickly
•-Incremental
•-Flexibility
•-Adaptive
•-way of life
•-Value driven

What is ‘Testing in Agile’?

Since, we have briefly touched upon the two terms separately, we may say that ‘Testing in Agile’ corresponds to the-
•Testing activities performed in the Agile environment
Software testing activities do not only mean manual testing. But these activities, cover all or some of common types of testing conducted for any project. So, some of the popular testing activities performed under Agile environment are:
•Unit Testing
•Functional Testing
•Regression Testing
•Performance/Load/Volume Testing
•Automated Testing

Topics in this series

Testing in Agile
Approach to Agile in Testing
Challenges for Testing in Agile
Advantages of Testing in Agile
Principles of Agile Testing
Best Practices in Agile Testing
Agile Testing Myths

Principles of Agile Testing through the Agile Manifesto

by

I am working on the Agile Testing article and had a hard time when it came to principles of Agile Testing. All the bloggers and websites have done a good job in listing various pointers as the principles. In this post, I have tried to understand the Principles behind the Agile Manifestothrough QA or Software Testing perspective:

1.    Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Each sprint in Agile must be a deliverable which satisfies the customer’s requirements. The value of the same is always appreciated much more when it’s bug free thus making through testing at each stage a mandatory. Regression testing is one area which generally takes a lot of time as the software starts evolving in later sprints. Automation of regression and unit test cases must be taken into consideration right from the beginning to curb this issue.
2.    Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
Changing requirements leading to issues in related functionalities is generally the top reason of defects and thus all deliverables must be tested thoroughly before UAT. At the same time, testers also need to make sure that their test suites are updated regularly. Getting the regular review of test cases from the users is also recommended to make sure the same is updated as expected.
3.    Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
Many times, delivering a working product in shorter timescale leads to skipping the testing activities. This is one area; the teams need to sit together and find the possible solutions. Automated unit tests and regression tests combined with exploratory testing may be a savior in such situations. Automation does come along with maintenance cost but the same can drastically reduced when right automation framework with maximum shared objects and repositories is used thus making the individual scripts easy to maintain.
4.    Business people and developers must work together daily throughout the project.
Agile is all about evolving requirements. I think strong communication comes into picture in this principle. All the stakeholders: Development team, QA team, users/business analysts etc must be an integral part of core team and included in all communications in all cycles.
5.    Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
QA team in Agile are often ignored due to various factors – time being the most common. Just like all teams, the test team also need to be motivated and provided with required environments to perform their testing activities effectively.
6.    The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
This is one principle QA team also need to follow and understand properly. While there is a process for all the activities including raising bugs, but at the same time discussing the same with teams and providing their feedback on various details is equally important. Where-ever possible, all collaborating teams should be physically located together to save time and effort on emails and calls.
7.    Working software is the primary measure of progress.
All stakeholders must understand that a working software is primary goal of any project. The QA team should also focus more on getting the product deliverable instead of concentrating on breaking the system (which is the general mindset of testers). The data generated should be simple and easily understood by all stakeholders. Complicated reports with fancy metrics might look great on display but they do not guarantee client satisfaction. The only full proved method for client approval is a working software as per customer expectations.
8.    Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
The QA team should be included in all details and have the crucial responsibility of making sure that the test coverage is achieved to highest possible.
9.    Continuous attention to technical excellence and good design enhances agility.
The QA team must ensure that all testing activities starting from understanding and test case writing to execution and bug verification are performed properly. The development team also need to make sure that all inputs of test team are taken seriously. It’s is the joint responsibility of QA and development team which ensures excellence and quick delivery of quality products
10.  Simplicity–the art of maximizing the amount of work not done–is essential.
When I try to relate this with testing, ‘Unit Testing’ is the first term which comes to my mind instantly. Unit testing is the most basic of all and is often ignored the most. Making sure that the smallest unit of code is working fine is the first step to quality software.
11.  The best architectures, requirements, and designs emerge from self-organizing teams.
When I think of this principle with respect to testing, All the basics of testing seems to apply here. The QA teams need to ensure that the requirements are covered in the test suites, regression tests are regularly updated, right tools for automation are chosen and the team members are skilled enough to cope up with frequent changes in the requirements.
12.  At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

               The more frequently product is tested for its effectiveness, the more robust it becomes. Regression testing can play an important role while executing this principle.

See Also:

 
I tried my best to explain the principles with respect to QA or software testing mind. Please feel free to provide your inputs. I would be more than happy to edit or update any of the points.