Test automation promises many attractive benefits such as running multiple tests overnight at a click of a button, eliminating mindless work, increasing test coverage, and reducing the cost of testing. So the question is, how do we get started?
The first order of business is to be sure that test automation is the right decision in your particular case. The initial automation effort can be expensive, especially if the systems have not been designed with test automation in mind (and very few of them are). Before considering automation make sure you can positively respond to ALL the following pre-requisites:
For an effort to be considered for test automation, it should result in a positive ROI. Since the expected lifespan of testing a system is not always known or can be hard to predict, an attempt at a comprehensive ROI calculation at an early stage will likely miscalculate the actual ROI. A short term ROI estimate (between one to three years) is safer and easier to calculate, typically utilizing at least some of the following factors:
An ROI calculation is not enough, though; as indicated above you must also ensure that you have a solid test process in place including things like scope, strategy, test types and test levels. Simply put, automated garbage is still garbage, so if you haven’t been able to positively answer the entire automation pre-requisites list above, start there. Also, keep in mind that test automation will almost never replace the first round of manual testing for new functionality but is rather targeted at reducing the cost and risks associated with regression testing. There are cases in which a progressive automation approach enables utilizing test automation for both regression testing and testing of new functionality, but it is not recommended for organizations just getting started with test automation.
Once your foundation and preliminary ROI calculations are in place, it is time to run a test automation proof of concept (POC) project.
A POC project should start with defining the immediate goals it will endeavor to meet. While doing so, keep in mind that the POC’s main purpose is not actual automation implementation, but is rather intended to help determine what kind of methods and resources would be required for a successful automation implementation. It is recommended that the POC project take no longer than three to four weeks. It should have a balance between the need to discover the main automation challenges and demonstrating feasibility with the need to support relatively quick decision-making. In addition to organization-specific goals, your list of POC goals should include at least some of the following:
Once your POC goals are in place, start the POC project by educating yourself on available automation approaches and their advantages. Selection of test automation approaches should include reviewing at least some of the following factors:
When an approach is selected, next consider the tools you will need. Some of the important factors are:
Once the tools have been chosen, the next step in a POC project is to establish a detailed Test Automation Implementation Plan. The plan should include a list of reusable business and utility functions to be tested, as well as a detailed automation breakdown. The plan should allow for the distribution of workload, estimation of the timeline, and tracking of progress.
The last topic to consider is technical guidelines, such as: object mapping structures, creation and maintenance processes, and coding conventions. When all of the preliminary work is finished, it is time to actually implement the selected scenarios on the tools of choice. From these scenarios, a list of encountered and predicted challenges should be generated and solutions for said challenges evaluated. If the challenges prove to be insurmountable, the POC conclusion is that there is no test automation feasibility. Otherwise, the information gathered should enable meeting the POC goals and enable more accurate recalculation of the ROI. If a revised ROI is unacceptable, another POC with an alternative approach or toolset can be performed. Alternatively, this could indicate that test automation might not be a good idea for this system.
If the POC is completed successfully, and the decision to pursue automation is reached, the concern of automation maintenance should become the leading factor throughout your test automation implementation as it will be the major factor in delivering positive ROI. To ensure efficiencies, it is important to build a framework that supports minimal and easy maintenance.
A good maintenance process follows this general outline:
In this paper, we outline the basic steps of getting started with test automation at your organization. While each organization has different needs and technologies, we find that just about every organization can benefit from utilizing test automation and that the above step by step guide will provide a good framework for getting started.
There is one last thing to keep in mind: when looking at test automation, the following list of areas typically benefit from test automation:
