by Hemamalini Senthilarasu

Software testing plays a major role in the software industry with more and more organizations understanding its part in producing quality software. As the number of business requirements increase, there is an increased pressure to deliver software products with fewer resources in less time and of higher quality. Maintaining high quality demands extensive software testing before product releases, and testing on various supported platforms and scenarios to ensure that the product performs well. As software application grows, the number of scenarios to be tested grows so does the number of test cases. This eventually causes the test suite maintenance to be hard and execution to be time consuming. This can be solved by adopting test automation, which is the execution of sequence of actions without human intervention. Test automation eliminates manual errors in execution and comparison and provides faster results. It also saves cost as a result of time saved by running automated tests rather than manual tests.

Test automation is supported by means of an automation framework. A test automation framework can be defined as the set of assumptions, concepts and practices that enhances maintainability and reusability of automated test scripts. It defines a standard that can be followed in an organization. There are several types of automation frameworks that are currently used in the industry.

• Data driven framework:
  This framework enables re-use of the test script with different sets of data. For example, when testing the login page of the application, sometimes we might want to test the login process with multiple sets of user id and password. A data driven framework can be used in this case so that the same script is used to test different login details that are obtained from a database. This helps to reduce the time spent in creating automated test cases to test each set of user id and password and thereby enhances maintainability.

• Keyword driven framework:
  In keyword driven frameworks, test cases are expressed as a sequence of keyword driven actions. A driver script executes the actions, which calls functions as prompted by keywords. A keyword driven framework helps to create test cases by describing each step of the automation instead of writing a script to go through each step. For example, consider the same login process of the application as described earlier. Traditionally testers write a script to navigate to the application, enter the user id and password and click the submit button. With keyword driven testing, each of these steps is described as an event. The testers need not understand the scripting language and it is much simpler to automate than take the traditional approach.

• Modular automation framework
  Modularity driven frameworks require the creation of small independent test scripts that represent modules, sections, and functions of the application being tested. These small scripts are then used in a hierarchical fashion to construct larger tests realizing a test case. This type of framework applies the principle of abstraction or encapsulation to improve the maintainability and scalability of automated test suites.

• Hybrid automation framework
  Hybrid automation framework uses a combination of three different frameworks: data-driven framework, keyword-driven framework and modularity-driven framework.

A test automation framework is built on top of the internal architecture of the automation tool. It defines a standard for implementation and ensures structured design and organization of automation test scripts. A test automation framework must support faster test script generation, enhance maintainability, enable reusability, modularity and should be leveraged across multiple projects. In this blog, I have detailed some of the elements that should be considered while designing an automation framework.

• Function libraries:
  A test automation framework should be application independent so that it can be used for all applications. This is supported by using a function library that contains the generic set of reusable functions that are common for all applications. The function library should contain application independent functions that include reporting, error and exception handling. Some of the common functions that can be created for a web application are launching a new instance of a browser, verifying a text on a web page with error handling, entering data into a textbox, writing log to a file, and so on. The function library is the only component in the framework that has to be modified if the application has been changed.

• Driver script:
  Driver script is a script, which drives the execution of test scripts sequentially and updates the execution results to a database.

• Global variables:
  Global variables that are used across all automated test cases should be defined and placed in a common repository and should be provided as an input to the automation framework. Once the variable is defined, we can use the variable instead of the entire object hierarchy, which makes the test script easier to read and maintain. The global variables can also be used to store runtime values and used in another script.

• Communication with data storage:
  An automation framework should have a means to communicate with different types of data storage such as excel file, CSV file, xml file, database, etc in order to support data driven testing.

• Naming convention:
  Naming conventions should be implemented for test scripts, function names, variable names and object names so as to keep the automation framework maintainable and consistent, regardless of the individual tester developing the script or executing the test.

• Ability to run on multiple hosts:
  It is necessary for an automation framework to run on multiple platforms as there may be a need to test an application that is supported on multiple platforms.

• Ability to organize tests:
  It should be possible for an automation framework to group a set of test cases that are related. For example, test cases related to login processes can be grouped under a group named 'Login'. Test cases can also be organized in a hierarchical fashion in order to give a better view on where the component or module lies.

• Ability to create batch test runs:
  Once the test cases have been grouped, it should be possible to run all the tests in batches sequentially with some delay introduced before each test case if needed.

• Minimize test script dependencies:
  The test scripts should be created with minimal dependencies and structured in such a way that when executing the scripts sequentially, even if some test cases fail, it does not affect the other test cases.

• Logs:
  One of the important elements in an automation framework is writing logs to track the execution of the test case. A log file can be created for each application when the corresponding test cases are executed. It is very important to generate debug information at various points in a test case and log file should hold important information about a test case such as the functions it calls and success or failure of the function calls. This helps to find the problem quickly when a test case execution fails.

• Ability to schedule execution:
  Once the automated test cases have been developed, they can be scheduled to run unattended on different hosts at different intervals, and in recurrence.

• Ability to report execution results:
  Execution results of the automated test cases must be reported with various details important to the tester, such as the test name, test location, result (pass or fail), errors, warnings, execution time (time to run the test case), host name, user name, etc. The contents of the report should be customizable depending on the tester's requirements and it should be easily readable.

• Integration with mail server:
  The test automation framework can be integrated with mail servers such as Microsoft Outlook so that once the execution of the automated test cases is completed, the result report can be sent via email to the appropriate tester responsible for the test cases.

• Integration with configuration management tools:
  Test scripts and data files should be stored in a centralized repository so as to track and control changes in the scripts and data when more than one tester manipulates them at the same time.

• Integration with test management tools:
  There are two ways in which a test management tool can be integrated with the test automation framework.
  • A test management tool can have the ability to hold the location of the automated test cases and integrate with an agent of the automation framework. When the automated test cases are executed in the test management tool, the agent calls the driver to execute test cases in the automation framework. Once the test cases have been executed, the driver passes the results back to the agent, which then passes it back to the test management tool. The test management tool can then display the results in the format configured by the tester.
  • Test management tools help to manage manual test cases and produce the results in various formats. Automation frameworks can be integrated with the reporting section of the test management tool so that once the automated test cases have been executed the result data can be combined with the manual test results. This helps to consolidate and analyze the manual and automation test results.

It is essential to implement an effective and reusable test automation framework in order to keep up with the pace of product development and delivery. This can be realized by considering the elements described above to build a robust test automation framework. When testing an application using a robust framework, the quality of the resulting application can ultimately be enhanced.