Automation Framework: Keyword Driven Framework

Keyword Driven framework is also known as table-driven or action-word driven framework. In this framework, an application map is defined including it’s objects, components, actions and an expected value for each component. It requires very deep knowledge of applications and automation testing expertise.

Keyword-driven tests look very similar to manual test cases. In a keyword-driven test, the functionality of the application-under-test is documented in a table as well as in step-by-step instructions for each test. In this method, the entire process is data-driven, including functionality.

Merits

  • The Detail Test Plan can be written in Spreadsheet format containing all input and verification data.
  • Application, language, tool independent.
  • Takes time, but most successful in long run.
 Demerits 

  •  The Development of “customized” Functions and Utilities requires proficiency in the tool’s Scripting language.
  • Takes lot of time iitially.

Automation Framework: Data Driven Framework

Data Driven Framework is a systematic approach to test an application. In data-driven framework test input and output values are read/write from/to data files (Mostly Excel files) and are loaded into variables in captured or manually coded scripts. In this framework, variables are used for both input values and output verification values. Navigation through the program, reading of the data files, and logging of test status and information are all coded in the test script.

Merits of Data Driven Framework:-

  • Scripts may be developed while application development is still in progress.
  • Utilizing a modular design, and using files or records to both input and verify data, reduces redundancy and duplication of effort in creating automated test scripts.
  • If functionality changes, only the specific “Business Function” script needs to be updated.
  • Data input/output and expected results are stored as easily maintainable text records.
  • Functions return “TRUE” or “FALSE” values to the calling script, rather than aborting, allowing for more effective error handling, and increasing the robustness of the test scripts. This, along with a well-designed “recovery” routine, enables “unattended” execution of test scripts.

Demerits of Data Driven Framework:-

  • Requires proficiency in the scripting language used by the tool (technical personnel).
  • Multiple data-files are required for each Test Case. There may be any number of data-inputs and verifications required, depending on how many different screens are accessed. This usually requires data-files to be kept in separate directories by Test Case.
  • Tester must not only maintain the Detail Test Plan with specific data, but must also re-enter this data in the various required data-files
  • If a simple “text editor” such as Notepad is used to create and maintain the data-files, careful attention must be paid to the format required by the scripts/functions that process the files, or script-processing errors will occur due to data-file format and/or content being incorrect

Automation Framework: Modular Automation Framework

A Modular Automation Testing Framework is also one of the most basic type of automation framework. In Modular Automation Framework, each business module/functionality is separated out and handled independently. It is also known as ‘Test Script Modularity Framework’.

The Modular Automation Testing Framework requires the creation of small, independent scripts that represent modules, sections, and functions of the application-under-test. These small scripts are then used in a hierarchical fashion to construct larger tests, realizing a particular test case. 

 Advantages:

  • It reduces test Automation time.
  • Performance is high than recording and playback.
  • Execution and debugging is possible while you are building a component.
Drawbacks:
  • Less usage of keywords,no concentration on Data driven testing and low performance than latest Frameworks like Keyword Driven Framework..

Types of Modular driven Framework

  1.  Test Script Modularity Framework: Enables creation of Small, Independent Scripts representing  Modules & Functions of the Application under Test (AUT).
  2. Test Library Architecture Framework: Enables creation of Library Files representing Modules & Functions of the Application under Test (AUT)

Automation Framework: Linear Automation framework

Linear framework is the most simple of all the frameworks. Linear framework has individual scripts for each test case and is executed individually. It depicts the basic record and playback approach. Linear Framework is suitable to convert small to medium size manual scripts into equivalent automation scripts – one to one basis.

Advantages of Linear framework:-

• Quick development – There is little planning or consideration of the various Quality Attributes when creating Linear scripts. As the scripts are created by recording, it does not require much effort.

• Doesn’t need through knowledge of the tool – Test automation typically requires through knowledge of the automation tool features and languages supported. One should also be aware of how the tool interacts with the application under test. The use of Record and Playback in the creation of Linear scripts dynamically creates code that replicates actions performed in the application.

• Script Independence – The scripts are independent of each other.

Disadvantages of Linear framework:-

• Issues in Playback – As the linear framework rely heavily on Record and playback, the recorded scripts do not playback properly for various reasons.

• Duplicity of work – Linear framework proceeds on individual scripts, thus reuse feature is not used in it. If some scripts require same action to be performed, it would again be recorded in the test.

• Scripts are difficult to read – As the script is auto generated, it has no comments and thus makes it difficult to read and interpret.

Automation Framework: Types of Automation Framework

Automation Frameworks can be classified into following types: 

  • Linear Framework
    • Linear framework is the most simple of all the frameworks. Linear framework has individual scripts for each test case and is executed individually.
  • Modular Framework
    • In Modular Framework, the test is divided into reusable functionalities also called as actions, so that they can be used in any test.
  • Data Driven Framework
    • Data driven framework is a systematic approach to test an application. In data-driven framework test input and output values are read/write from/to data files (Mostly Excel files) and are loaded into variables in captured or manually coded scripts.
  • Keyword Driven Framework
    • Keyword Driven framework is also known as table-driven or action-word driven framework.In this framework, an application map is defined including it’s objects, components, actions and an expected value for each component.
  • Hybrid Automation Framework
    • Hybrid framework is the combination of all of the above techniques, pulling from their strengths and trying to minimize their weaknesses.

Automation Framework – Advantages and Development Challenges

Advantages of Automation Framework:-

  • Low cost for maintenance
  • If there is change to any test case then only the test case file needs to be updated and the Driver Script and Startup script will remain the same. There’s no need to update the scripts in case of changes to the application.
  • Standard scripting that will result in team consistency during test library design and prevent individuals from following their own coding standards, thus avoiding duplicate coding.
  • Test Automation scripts are separated from input data store and only the input data gets manipulated whereas no modification is needed for the test scripts.
  • By developing the libraries, they can be reused again and again, saving time for the entire organization/project team.
  • By giving right role-based access, the standard process of Test Automation scripting can be maintained.
  • Future Enhancements Support
  • Ensures consistency of testing.

 Test Automation Framework Development Challenges: 
  • Cear vision of what needs to be achieved out of this automation. It should address core questions like testing model, types of testing, which areas need to be automated etc.
  • Tool identification and Recommendation process is a crucial process, as it means considering critical factors like creating a standard tool evaluation checklist, types of testing, and acquiring multiple tools to perform different types of testing.
  • Framework design involves identifying requirements from multiple areas like identification of necessary utility/components, types of input data store to be communicated, communication between the systems and utility/component development, etc.
  • In order to have any chance of building a sustainable solution the Test Automation Engineer will have to a close working relationship with the development or product support – if the application has been purchased by a third party vendor.

For designing a framework, various elements need to be taken into consideration. Some of them are:

  

• Actions to be performed
• Database Communication
• Communication with additional automation tools
• Device Communication
• Log
• Error Handlers
• Custom Messages

Based on these, Test automation framework would be designed using the following guidelines like.

• Application-independent (if possible). 
• Encapsulate the testers from the complexities of the test framework.
• Identify and abstract common functions used across multiple test scripts.
• Decouple complex business function testing from navigation, limit-testing, and other simple verification and validation activities.
• Structure scripts with minimal dependencies – Ensuring scripts executing unattended even on failures.

Automation Framework

A test automation framework can be defined as a compilation of assumptions, concepts, and practices that provide support for automated software testing.

Framework comprises support programs, processes, code libraries etc to help develop and glue together the different components in a project.

Physically, Frame work is nothing but a collection of folder structure and placing all files into corresponding folders.

Problems with Record/Playback Process:-

A general conception about automation tools is that they can be used easily by using their record and playback feature. The process followed in record and playback (also called as Capture/Replay) approach is:-

  • Record the user actions using the capture/record feature of the automation testing tool. 
  • It results in an auto generated script. 
  • Complete the test cases by adding verification points in the script.  
  • Add delay and wait state points where the different actions need to be synchronized. 
  • Playback the scripts and log the results in a test management tool. 

The above process results in following issues:-

  • The script contains hard coded values which must be changed every time there is some change in the Application. 
  • The scripts are not reliable. Even if the application does not change, the scripts often fail on replay because of unhandled scenarios which were not recorded. 
  • If the tester makes an error entering data, etc., the test must be re-recorded 
  • If the application changes the test must be re-recorded. 
  • This method turns out to be costly (time consuming) in long run.

Integration Testing

Integration Testing is is performed after Unit Testing. Integration testing is done when individual software modules are combined and tested as a group.Integration testing is the testing of combined parts of an application to determine if they function together correctly. The ‘parts’ can be code modules, individual applications, client and server applications on a network, etc. This type of testing is most useful to client/server and distributed systems.

Integration Testing can be classified in three different ways.

  • Big Bang Integration Testing
  • Top Down Integration Testing
  • Bottom Up Integration Testing
  • Hybrid Integration testing

What is Unit Testing?

Unit Testing is the testing of individual software components or modules. It is the foremost level of software testing and is commonly performed prior to integration testing.
Unit testing performed in the implementation phase can result in significant gains in software quality. If unit testing is done properly, testing phases done later on will be more successful. Unit testing is considered the job of developers only and test engineers need not know about Unit testing. But this is not the case, Unit testing is as important for test engineers as it is for developers.

Developers generally write the unit test cases but it is also important for the testers to understand them. Tester’s understanding of the framework and unit testing can certainly help him/her in designing the flow for unit test cases, generating test data and making good reports. All these activities will ultimately help a tester in the long run as product quality will increase significantly because of the efforts a tester put in on the unit testing.

Advantage/Benefits of Unit Testing

  • Unit Testing plays a major role in the total testing efforts.
  • It allows being able to test parts of a project with out waiting for the other parts to be available.
  • Achieve parallelism in testing by being able to test and fix problems simultaneously by many engineers.
  • Be able to detect and remove defects at a much less cost compared to other later stages of testing.
  • Be able to take advantage of a number of formal testing techniques available for unit testing.
  • Simplify debugging by limiting to a small unit the possible code areas in which to search for bugs.
  • Be able to test internal conditions that are not easily reached by external inputs in the larger integrated systems (for example, exception conditions not easily reached in normal operation).
  • Be able to achieve a high level of structural coverage of the code.
  • Avoid lengthy compile-build-debug cycles when debugging difficult problems.
  • Unit testing is more cost effective compared to the other stages of testing.

Demerits of Unit Testing:-

  • Testing can be monotonous, boring and repetitive at this level.
  • Poor Documentation of Test cases may be there.
  • Coding Drivers and Stubs.
  • Informal testing process.
  • Poor Regression Testing.
  • Lack of Complete Testing tools.

Gray box testing

Gray box testing is a software testing technique that uses a combination of black box testing and white box testing. Gray box testing is not black box testing, because the tester does know some of the internal workings of the software under test.

Gray box testing is a powerful idea. The concept is simple; if one knows something about how the product works on the inside, one can test it better, even from the outside. Gray box testing is not to be confused withwhite box testing ; i.e. a testing approach that attempts to cover the internals of the product in detail. Gray box testing is a test strategy based partly on internals. The testing approach is known as gray box testing, when one does have some knowledge, but not the full knowledge of the internals of the product one is testing.