Difference between revisions of "How to get started with automated testing"

Latest revision as of 05:06, 21 February 2014

The benefits of automating part of the tests are tangible and real. However, before starting to automate, some guidelines should be taken into account in order to guarantee the success of automated testing using gxtest over genexus applications.

Start with a pilot project

Just like with most software engineering practices, it is advisable to choose a suitable project and use it as a pilot project to get started.

Using a pilot project has several advantages:

• defining an initial methodology that is adequate for the way in which the company works.
• gaining experience in the use of automation tools.
• measuring benefits.
• assessing problems.
• having a project of limited scope in which to focus our efforts to improve quality.

Apply metrics

This pilot project should make it possible to show, in the shortest possible time, the ROI of applying automated testing. For this reason, some metrics should be selected (according to the reality of the company and the product) in order to showcase it. Some of the metrics that can be used to this end are as follows:

• hours of testing / number of errors found by the client
• number of errors found by the client / size of the system
• error detection time (time elapsed since the error occurred until it was detected)

Designate people in charge of performing automated tests

Once the project portfolio has been defined, the next step is to designate the people who will work on it to begin the automated testing experience. It is recommended that the same analysts who know the requirements and perform the traditional functional testing are selected to perform test automation. This task should be performed by the same group for several reasons: so that no competition is created between manual and automated testing; it helps ensure the adequate selection of tests to be made automatically; the automated testing tools can be used not only to automate test cases but also to generate data for those test cases.

How can the people in charge of testing be trained?

The learning curve of each person working on automation can vary depending on the level of their knowledge of automated testing and automation tools. It is advisable that when they start to receive training on the tool to be used (with GXtest users can start by following, for example, the GXtest Designer Tutorial), they also read material on the methodology and experiences of automated testing in general. The recommended reading includes the 4th Edition of the Testing Experience magazine, which focuses on automated testing.

Select the test cases to be automated

Once the project and the people who will participate in it have been selected, the test cases that will be automated have to be defined. The error most frequently made when starting to automate is trying to automate everything. Our recommendation is that you DO NOT TRY TO AUTOMATE ALL TEST CASES.

From a standpoint of the business and also the internal structure of the system (for which the developers have to be involved) we must decide which test cases will be automated. Some factors to take into account are as follows:

Which test cases will have to be run more times during the project? Typically, those test cases associated with the application core can be built at the beginning of the project and can later be used (adapting them as needed) for the rest of the system’s life cycle. Which test cases require many hours of human work and are very repetitive? Sometimes a test case demands tedious work to configure the test environment before running the test case. Automating these test cases is useful to free human resources so that they can focus on other, more challenging cases. What features are critical for the client/user? The features that must always work when released to the client are the ones that should be included in the automated cases. What’s the cost of automating a test case? Some test cases are very difficult to automate, whether because they require complex visual validations or for other reasons. That’s why if the cost of automating is too high, sometimes it's better to leave it for manual execution. Taking these characteristics into account, we can evaluate each test case to make a ranking or prioritize the test cases to be automated.

Automate!

After selecting a project for a pilot and choosing certain test cases to make them automatic, we recommend that you start with a limited subset of test cases (no more than ten) and put them to work. This means designing them (in paper, worksheets, etc.), making them automatic (with GXtest Designer and Recorder), preparing the environment and data to be able to test them, preparing the programmed execution of these cases (with GXtest Manager, setting them to be run every night, for example), and defining the work methodologies.

Once we complete this, we will start to see the benefits and populate the automated tests machine, making it grow and increasing its benefits.

See Also

• How to design test cases
• "Testing Experience" magazine 4th edition

• Content from Abstracta, Enlgish source: http://www2.gxtechnical.com/portal/hgxpp001.aspx?15,7,400,O,E,0,PAG;CONC;123;11;D;25328;1;PAG;MNU;E;25;2;13;1;MNU