Agile Testing Myths

Almost every existing process comes with it’s own share of understanding at individual level. Software Testing itself has always been a debatable topic. It is surrounded by many myths all of which have counter facts associated with them. Head on to the post – Software Testing Myths and Factsto read some of such misconceptions. Therefore, Agile Testing Myths are there out in open aswell.
Agile Testing Myths
Coming back to Agile Testing, below are common assumptions many teams have:

Agile means

ZERO documentation
Documentation is a critical process project teams run away from. With Agile in force, it is a generic response to get that – “Since we are following Agile, there is no documentation of requirements and other details”. Actually, If followed sincerely, The end result of an Agile project would result in much better and detailed documentation which would be well structured, easy to read and follow.
No planning
Ask an agile team for their project plan, the maximum they would be able to provide is the current or current + next sprint backlog. On the contrary, Agile process insist on having the sprint backlogs ready for current along with upcoming 2 sprints along with all the estimates and acceptance criteria. Thus, in practice, planning in agile projects is more rigorous.

Unit testing

Is sufficient and thus remove the need for manual testing
It is a common misconception that if Unit Testing is performed effectively, testing activities may be skipped. While it is true that unit tests play a significant role in reducing bugs, the fact that they cater to specific and limited scope must be considered as well.
Can be built into regression suite
In continuation to the point above, creating a regression suite from unit tests is also seen as a practice in some teams. Projects must understand that Regression Testing is altogether a separate activity with it’s own scope. While some of the unit tests would always be common to both regular test cases as well as Regression Test Cases, compiling all the unit tests in one place and calling it a regression suite is a complete no-no!


Is impossible
Test Automation in Agile development projects is sometimes considered an impossible task! The main reason for the same is lack of time. But then, I think, the remedy to this points to the very first myth – “Agile means No Planning”. The need of Test Automation and scope for the same must be a part of planning process from the very beginning.
Remove the need for manual tests
With automation in place, projects tend to avoid manual tests in respect of time and resources. Since, It is not possible to cover each and every case in automation, Manual Testing can never be replaced by Automation Testing. Even if the tests show 100% coverage in reports, exploratory testing generally done unintentionally as a part of manual testing would be at loss.

Testing is not a part of ‘Done’

It is quite typical with the development teams to mark a story done after implementation from their end even before it is tested. Sometimes, to get the burndown chart right, teams even move the user story to “Done” state if it has a bug associated with it. The explanation given is that the bug thus raised is already in backlog, so the user story should not move forward.
In ideal practice, any story is not “Done” until is has been tested successfully with no pending bugs associated with it.
More Topics in this series:
Testing in Agile
Approach to Agile in Testing
Challenges for Testing in Agile
Advantages of Testing in Agile
Principles of Agile Testing
Best Practices in Agile Testing
Agile Testing Myths

Best Practices in Agile Testing

In the previous posts, I covered basics of Agile Testing. Those include It’s definition, approach, challenges, Advantages and Principles of Testing in Agile. Here, is another very basic but important topic which is generally underrated – The Best Practices to follow while Testing in Agile environment.

       Unit test is a must

Unit Test – Smallest piece of code that can be tested individually.
This smallest piece of code is the one which is ignored most commonly. Test execution activities begin with Unit Testing and thus form the foundation of quality checks. Ironically, unit testing is the one which is most commonly ignored or cut short upon.
The teams must understand the importance of Unit Tests and make sure they are being executed. Automation is also another important factor which broadly help unit testing.

       Minimize dependencies

One of the most important attribute of an effective test case is that it can be executed by anyone in the team irrespective of who authored. The same principle applies to Agile too. The tests must be written in a manner that they have minimum dependency on test data and other configurations and thus can be executed and used to generate reports by anyone, anytime

       Follow Test Driven Development (TDD) or Behavior Driven Development (BDD) appropriately

TDD and BDD are the latest buzz words when it comes to boiling down another buzz word – Agile! While these techniques are definitely useful and prove very handy, care must be taken while implementing the same. It is not always necessary that the models fit into


     Test Automation is yet another field which is always in discussions and looks really attractive. It’s a different topic of discussion, but for now, below are the areas which can be good candidate for for automation:
       Unit Tests
       Regression Tests

       Create Scenario/Story-based Test Cases

      With the development being done in Agile and releases are done as a collection of user stories,  testing activities should not be ignored and follow similar procedures. It’s important to write test cases based on individual user stories which are further linked appropriately to the Epics.
Topics in this series:

Advantages of Testing in Agile

With Agile being the latest hashtag, more and more organizations and projects are taking steps to adapt the agile software development process. At the same time, most of the teams which are going agile or planning to move into the direction of Agile process are still reluctant to include Software Testing on the same page. Benefits of testing to projects are now widely known and all of them apply to Agile projects also. Below are some major points which can be re-emphasized when talking about the advantages of Testing in Agile.
Improved Quality
Quality is a wide term and without any doubt the responsibility of whole team which comprises not only testers, but also all other stakeholders. When testing is involved right from the beginning of project, provide valuable inputs to taken required actions to improve the quality of products.
Saves Resources
Budget – both in terms of time and money is the prime concern of clients and they consider it as main resource.
Early detection of bugs has been the mantra to minimize the effort of fixing the same. Needless to say, that when the test teams are involved continuously, helps the teams to progress in this direction. When there are less bugs in the applications or the products, clients see a drastic improvement in the utilization of their resources – time and money.
Improved Communication
Effective communication is the key to the success of any Agile project. Making testing as an integral part of the Agile Framework ensures that the team is aware of all(or at least most of them) possible scenarios and thus helps them achieve the target.
Since testers have a different mindset than developers, the whole team can thus work together and collaborate better. Also, the clients then adapt a better picture of the product and understand the details of functionalities rather than being caught in the scenario of coming back with a production issue when they face it.
These are the major Advantages of testing in Agile that I have come across. Do share the ones you think of too in comments.

Challenges in Testing in Agile

‘Software Testing’ itself has always been going through many challenges. After decades, the concept was now clear and teams have realized the importance of same. Now with the Agile approach being the buzz word, teams are already struggling to keep the procedures, templates and quality in sync. Agile based projects generally go through a lot of challenges by themselves and software testing is often neglected as a result. Below are the major challenges testing teams face while working in Agile teams:
       Changing Requirements
Frequent changes in requirements is the first thing which comes into picture when we talk about Agile environment. This feature is the first and major challenge QA teams face in projects facing Agile methodology. Since the requirements keep on changing, it becomes bit difficult to regularly update the test cases and ensure test coverage.
This can be overcome if the testers and developers are on the same page about progress modifications made in each other’s activities.
       User stories with no description
One of the myths around Agile methodology is ‘zero documentation’. It is this misunderstanding which leads to one liner user stories or stories with too less information.
This challenge again need inputs from both development and QA team. Developers need to understand that while Agile methodology discourages bog documentation around every detail, basic documentation of the functionalities is still mandatory and required. At the same time, QA team also need to accept the user stories with less information and try to take as much as they can on a high level from them. Such test cases can always be marked with a filter and updated as and when the information is available.
       Waterfall inside Agile
This is another major issue in many projects when the sprints become available to the QA team only towards the end of it. The testers then struggle hard in order to maintain test cases and performing other activities in a short period of time.
Making sure that the User Stories are available to all stake holders right from the time of their creation is very important for a successful Agile implementation.
       No time for Regression
Availability  of User stories to the test team close of sprint end leaves little time to execute regression test cases. The importance of regression testing is something everyone agrees these days. Thus, incomplete regression testing is one of the reasons of UAT defects.
Timely delivery of user stories is one way this can be rectified. Automation also plays major role in overcoming this challenge. Test Automation for unit tests and regression tests may significantly help.
       Communication gaps
If we look back, most of the challenges arise out of one or the other form of miss-communication. Merely formulating great processes and adopting latest methodologies do not guarantee the success of any project. The project is executed with high quality when all the teams and stake holders work together and help each other in every phase.

Topics in this series:

Testing in Agile
Approach to Agile in Testing
Challenges for Testing in Agile
Advantages of Testing in Agile
Principles of Agile Testing

Best Practices in Agile Testing
Agile Testing Myths

Approach to Agile in Testing

In my previous post, I talked in brief about “Testing in Agile“. Following the same, here are the approaches that may be followed for testing activities in Agile Framework.
Following the right approach to testing is a crucial factor to incorporate successful testing activities in Agile process. Since the development model is different than traditional methods, the teams need to sit together and finalize a process that best suits their project. Generally, following two approaches may be applied:
       Testing in Same Sprint
This method is like the traditional testing process where the QA team perform their test cases in the same sprint/Module (as per traditional process). Ideally, this should be the right approach to follow. Here, the QA team works on the same user stories as development team and prepare their test cases, perform execution iterations and make sure that the sprint delivers a quality product.
       Sprint lag – Sprint+1: Separate QA and DEV sprints
Since testing itself in Agile has its own challenges and tight deadlines is one of the main risk, most of the times, QA team is left with insufficient time to complete their execution iterations. This may happen due to continuous changing requirements which lead to last minute coding in development phase, deployment delays and many more factors.
To overcome this problem, some teams follow the ‘Sprint lag’ or ‘Sprint+1’ method. In this method, development and QA teams work on different sprints. For example,
Sprint 0:
Development Team – Works on Sprint 0
QA team – Works on test case preparation of Sprint 0
Sprint 1:
Development Team works on
1.    Sprint 1 development items
2.    Sprint 0 bug fixes
QA team works on –
1.    test execution iterations of Sprint 0
2.    test case preparation of Spring 1
Sprint 2:
Development Team works on
3.    Sprint 2 development items
4.    Sprint 1 bug fixes
QA team works on –
3.    test execution iterations of Sprint 1
4.    test case preparation of Spring 2

Testing in Agile

Anyone in the world of software development, is not new to ‘software testing’ in Agile environment. It basically means performing the various testing activities for applications or software or products which are developed under the Agile model.

In this post, I have tried to touch the topic on very broad scope. We will just go through the very literal meaning here.

 What is Testing?

 The process of evaluating a software application/project to detect and report differences between given input and expected output in order to make sure that it meets the customer requirements.
An activity or process which involves verification, validation of a software program to find out if it meets the business and technical specifications of requirements and providing a detailed defect report at the end of every iteration.

What is Agile?

Agile is the latest buzz word. Everyone is either adapting Agile Methodologies or want to go in the same direction. When we hear, or say Agile, following points come into our mind:
-Ability to move quickly
-way of life
-Value driven

What is ‘Testing in Agile’?

Since, we have briefly touched upon the two terms separately, we may say that ‘Testing in Agile’ corresponds to the-
Testing activities performed in the Agile environment
Software testing activities do not only mean manual testing. But these activities, cover all or some of common types of testing conducted for any project. So, some of the popular testing activities performed under Agile environment are:
Unit Testing
Functional Testing
Regression Testing
Performance/Load/Volume Testing
Automated Testing

Topics in this series

Testing in Agile
Approach to Agile in Testing
Challenges for Testing in Agile
Advantages of Testing in Agile
Principles of Agile Testing

Best Practices in Agile Testing
Agile Testing Myths

Principles of Agile Testing through the Agile Manifesto

I am working on the Agile Testing article and had a hard time when it came to principles of Agile Testing. All the bloggers and websites have done a good job in listing various pointers as the principles. In this post, I have tried to understand the Principles behind the Agile Manifestothrough QA or Software Testing perspective:

1.    Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Each sprint in Agile must be a deliverable which satisfies the customer’s requirements. The value of the same is always appreciated much more when it’s bug free thus making through testing at each stage a mandatory. Regression testing is one area which generally takes a lot of time as the software starts evolving in later sprints. Automation of regression and unit test cases must be taken into consideration right from the beginning to curb this issue.
2.    Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
Changing requirements leading to issues in related functionalities is generally the top reason of defects and thus all deliverables must be tested thoroughly before UAT. At the same time, testers also need to make sure that their test suites are updated regularly. Getting the regular review of test cases from the users is also recommended to make sure the same is updated as expected.
3.    Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
Many times, delivering a working product in shorter timescale leads to skipping the testing activities. This is one area; the teams need to sit together and find the possible solutions. Automated unit tests and regression tests combined with exploratory testing may be a savior in such situations. Automation does come along with maintenance cost but the same can drastically reduced when right automation framework with maximum shared objects and repositories is used thus making the individual scripts easy to maintain.
4.    Business people and developers must work together daily throughout the project.
Agile is all about evolving requirements. I think strong communication comes into picture in this principle. All the stakeholders: Development team, QA team, users/business analysts etc must be an integral part of core team and included in all communications in all cycles.
5.    Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
QA team in Agile are often ignored due to various factors – time being the most common. Just like all teams, the test team also need to be motivated and provided with required environments to perform their testing activities effectively.
6.    The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
This is one principle QA team also need to follow and understand properly. While there is a process for all the activities including raising bugs, but at the same time discussing the same with teams and providing their feedback on various details is equally important. Where-ever possible, all collaborating teams should be physically located together to save time and effort on emails and calls.
7.    Working software is the primary measure of progress.
All stakeholders must understand that a working software is primary goal of any project. The QA team should also focus more on getting the product deliverable instead of concentrating on breaking the system (which is the general mindset of testers). The data generated should be simple and easily understood by all stakeholders. Complicated reports with fancy metrics might look great on display but they do not guarantee client satisfaction. The only full proved method for client approval is a working software as per customer expectations.
8.    Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
The QA team should be included in all details and have the crucial responsibility of making sure that the test coverage is achieved to highest possible.
9.    Continuous attention to technical excellence and good design enhances agility.
The QA team must ensure that all testing activities starting from understanding and test case writing to execution and bug verification are performed properly. The development team also need to make sure that all inputs of test team are taken seriously. It’s is the joint responsibility of QA and development team which ensures excellence and quick delivery of quality products
10.  Simplicity–the art of maximizing the amount of work not done–is essential.
When I try to relate this with testing, ‘Unit Testing’ is the first term which comes to my mind instantly. Unit testing is the most basic of all and is often ignored the most. Making sure that the smallest unit of code is working fine is the first step to quality software.
11.  The best architectures, requirements, and designs emerge from self-organizing teams.
When I think of this principle with respect to testing, All the basics of testing seems to apply here. The QA teams need to ensure that the requirements are covered in the test suites, regression tests are regularly updated, right tools for automation are chosen and the team members are skilled enough to cope up with frequent changes in the requirements.
12.  At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

               The more frequently product is tested for its effectiveness, the more robust it becomes. Regression testing can play an important role while executing this principle.

See Also:

I tried my best to explain the principles with respect to QA or software testing mind. Please feel free to provide your inputs. I would be more than happy to edit or update any of the points.

Software Testing Abbreviations and Acronyms

Below is a list of Acronyms or Abbreviations I could collect from various sources on Google related to Software Testing

Application Binary Interface
Attribute Component Capability
Artificial Intelligence Applications Testing
Application Lifecycle Management
Average Method Complexity
agile model-driven development
American National Standards Institute
Application Programming Interface
American Standard Code for Information Interchange
Automated Software Test Framework
Acceptance Test Procedure
Automated testing lifecycle methodology
Application Under Test
Business Driven Development
Bit Error Test (Diagnostic Tests)
Built-in self-test (Diagnostic Tests)
Browser Integrated Test Environment
Bug fix verification
Business Requirement
Business Requirement Document
Business Requirement Specification
Boundary Value Analysis
Computer-Aided Software Engineering
Computer Aided Software Testing
configuration control board
Critical Design Review
Critical Error
Challenge Handshake Authentication Protocol
Configuration identification
Configuration management
Capability Maturity Model
Capability Maturity Model Integrated
Configuration Management Plan
Configuration Management Tool
Cost of Failure
Cost of Quality
Commercial Off-The-Shelf Software
Change Request
Create, Read, Update, Delete
Critical Testing Processes
Defect Detection Percentage
Data Flow Diagram
Document Object Model
Dynamic Software Development Method
Defect Removal Efficiency
Equivalence Partitioning
Entity Relationship Diagram
Estimated Time of Arrival
Extract Transformation Load
Frequently Asked Questions
Functional Design Document
Feature Driven Development
Failure Mode and Effect Analysis 
Function Point Analysis
File Transfer Protocol
Function verification test
Graphical User Interface
Google Test Analytics
Hierarchy, Input, Processing, Output
High Level Design
Hewlett Packard Quality Center
Interface design document
Integrated Development Environment
Initiating, Diagnosing, Establishing, Acting & Learning
Interface design language
Institute of Electrical and Electronics Engineers
International Organization for Standards
Internet Security Association and Key Management Protocol
Integrated Services Digital Network
Integrated Software Lifecycle Environment
International Standard Testing Quality Board (Certification)
Joint Application Development
Java Database Connectivity
Joint Technical Committee 1
Knowledge-Based Software Assistant
Thousands of lines of code
Knowledge Management
lower control limit
Linear Code Sequence And Jump (a software analysis method)
lines of code
Long-Sequence Regression Testing
model-driven development
Mean Time Between Failure
Mean Time To Fail
Mean Time To Repair
Mean time to critical failure
Minimum Viable Product
National Cyber Security Alliance
Non-Disclosure Agreement
Nonfunctional requirements
National Institute of Standards and Technology
National British standard
Operational configuration management
Open Database Connectivity
Objectives and key results
On Line Analytical Processing
On Line Transactional Processing
Object Relationship Diagram
Operating System
Open Systems Interconnection
Physical Audit
Performance and Coverage Analysis
Preliminary design review
Program Evaluation and Review technique Diagram
Post Implementation Review
Problem and Change Request Tracking System
Platform-Independent Model
Platform-independent model
Proof of Concept
Probability of Failure
Poor Obfuscation Implementation
Power- On Self – Test (Diagnostic Tests)
Platform-Specific Implementation
Problem trouble report
Quality Assurance
Quality Control
Quality Management System
Quality of Service
Quality Evaluation System
Rapid Application Development
Real Application Testing
Root Cause Analysis
Request For Change/Comments
Return on Investment
Reflexive User Interface Builder
Request for Proposal
Remote Method Invocation
Record and Playback framework
Reverse Semantic Traceability
Requirements traceability matrix

Structured Analysis
Systems Analysis and Design Technique
Static Code Analysis
Security Checklist
Source Code Analyzer
Standards committee
Software Change Request
System and software Design Document
Software Development Kit
Software Development Life Cycle
Software development plan
Software Engineering Institute
Search Engine Optimization
System Investigation Report
System Integration Testing
Service Level Agreement
Software life cycle
Service Oriented Architecture
Structural Query Language
Software quality system
Software quality system plan
Specification Requirement Document
Software requirements review
Software Requirement Specification
Software Maintenance and Regression Test System
Secure Shell
Software testing automation framework
Systematic Test and Evaluation Process
Structured Testing and Requirements Tool
Software trouble report
System trouble report
Subscriber Identity Module
State Table
System Testing
Software Testing Life Cycle
System Under test
Software Usability Measurement Inventory
Test Coverage Analysis Tool
Trusted Computing Base
Test Driven Development
Test filelData Generator
The Transport Layer Security
Test Maturity Model
Target of Evaluation
test point analysis
Test Process Improvement
Time Partition Testing
Total quality control
Testing and Test Control Notation
Test Traceability Matrix
test readiness review
Test Summary Report
User Acceptance Testing
upper control limit
unit development folder
Universal Description, Discovery and Integration
UML Testing Profile
User-based Security Model
Usability-Test Candidate
User Interface
Unified Modelling Language
Uniform Resource Identifier
Uniform Resource Locator
V & V
Verification and Validation
Virtual Environment
Virtual machine
Virtual Users
Work In Progress
Weighted Shortest Job First
Extensible Markup Language
eXtreme Programming
Cross Site Scripting

Software Testing – Myths and Facts

“Software Testing is relatively new” – I get irritated whenever I hear or read this. As if before the term, software’s being developed were not tested. Oh, I used the relatively new word again – checked for their expected behaviour.

Image source:
So, when we talk about myths about software testing, the above stands on top. There are few more I have tried to jot down in no particular order. Let’s see if you agree.

1.  Software Testing is relatively New
I do not need to write anything for this. We get to hear this ironic term at all places – big or small.

Fact:  It has been existing ever since first piece of code was written.

2.  Testing is boring
Testing being a monotonous activity is reflected very frequently at almost every platform. People have a misconception that testers keep on doing the same old clicks and data entry without any creativity.

Fact: A statement somewhere on internet says it all – “Testing is like sex. If it’s not fun, then you are not doing it right.”

3.  Testing is easy/No formal training is required
Testing being an easy job is another big misconception. There are arguments all over the world that since users keep on finding bugs, it’s no big deal being a tester. Sometimes, a proper planned formal training for the same is also neglected.

Fact: Many times testing proves to be more complex and tedious than development as its difficult to analyze the behavior without code (I am not saying that developers have an easy job, but access to code gives an edge on analyzing the behavior), Testers require a deep understanding of testing methods, business requirements and the development process.
Formal training and experience makes a good tester with solid skills or we can say a valued resource.

4.  Anyone can test
I cannot forget the statement by one of my managers – “Amita, don’t fell offended, but the fact is that testing can be done by anyone. I mean, you need special skills and technology knowledge to develop the software, but testing can be performed by developers as well.”

Fact: If that was the case, no company would have paid a dollar to testers. There would have been no companies that specialize in only testing services. This is the trap management falls into.

5.   A tester’s job is to find “ALL” bugs
100% test coverage is one of the top goals of most of the testers. This is the most common myth which clients, Project Managers and the management team believe in.
Fact: This is the trap management fall into. The very first principle of software testing states that – “Testing shows presence of defects”. Testing shows that defects are present, but cannot prove that there are no defects. Even if no defects are found, it is not a proof of correctness.
6.   Automated Scripts replace manual testing
Whenever I give a POC for automation, people have an impression – “Wow! Just click and all done!” Statements like – “So, if you create the scripts once, we can just run them for next releases”. This is the most common misunderstanding that automated tests are equivalent to manual tests. The worst is that there are testers and test managers who actually believe it.

Fact: No test automation tools can ever replicate human feelings or emotions.
For example, a tool can test the fonts, color and layout of a screen is as per the test script but it can never analyze if a screen is user friendly.
7.    Software Testing is “time consuming” and “expensive”.
I personally get the emails to revise our estimates for testing efforts very frequently. Sometimes, I hate to justify the minute details of my estimates. The considered myth that its too much effort – “time consuming” automatically generates the next one – “Expensive”.

Fact: Software testing principle 3 – Testing activities should start as early as possible in the software or system development life cycle and should be focused on defined objectives.
One cannot expect the testers to be magician and test the code overnight when they have no idea of background.
Decide – Pay during development or pay latter for production issues and reputation.
8.  Software Testing is same as Quality Control
I can bet that almost 90% people confuse testing with quality control. Software Testing, Quality Assurance and Quality Control are the terms used almost interchangeability.

Fact: Testing is just a component of software quality control, wherein a tester identifies the bugs for the stakeholders. Quality Control include many other activities like Reviews – Self review, peer review and structured walk through

9.  No changes – Regression testing is not needed
Regression Testing is skipped many times to save efforts. There has been no change in this code is the most common excuse to that.

Fact: Even if the module was not touched, spending some bucks on re-verifying the results may save the cost of an unidentified bug or scenario.

10. Software Testing is the career choice for failures
Fresher’s dread the software testing field. They think that it does not offer career growth and is a low profile job.

Fact: Think again. Facebook, Microsoft etc. pay millions to find a bug.

These are the most common myths I have faced during my career. If you have any additions, or faced similar experiences, please feel free to comment or share. This does not do not welcome general like/dislike comments. I love them the most… 🙂