Difference between revisions of "GXtest2OpenSTA for Performance Testing"

Revision as of 18:38, 2 July 2013

Introduction

GXtest2OpenSTA is a new feature that allows us to execute performance tests (stress testing, load testing, scalability testing, capacity testing) bringing down automation costs to approximately 20%. How? Even though GXtest is a functional testing tool, once we have automated our test cases we can automatically generate scripts for an open source performance testing software (OpenSTA), giving the tester numerous advantages, which we'll see further in this doc.

In this article we'll start with an introduction to performance testing, then we'll go through some general aspects of performance testing in OpenSTA, and we'll end up watching how GXtest2OpenSTA works, so we can see the true advantages that this tool provides us regarding the quality of our systems before we deploy them to the production environments.

Introducing Performance Testing

An essential point to consider about performance tests is that, for many reasons, it is impossible to test every functionality of an application. Having that in mind a new problem comes up: Which functionalities should we test? Which test cases should we select to automate and include in a performance test?

To solve these problems we can resort to historical data taken for the previous system (in case there is a previous version running), to get an idea of the active users, the number of times each functionality is used, etc. If we don’t have that possibility, we can always turn to our application’s expert’s knowledge (project managers, developers, users, or any other person who can estimate which functionalities are the most adequate to be included in our tests).

After we have selected which test cases to run, we have to automate them. Making the scripts is generally the most time consuming part of the project. Luckily, GXtest helps us in this task, reducing our efforts considerably.

When do we use performance testing?

Basically, there are two strategies we can use to approach performance testing. The traditional one is to start automating our test cases after the product is finished, so there won’t be changes in the source code. This strategy has the advantage of not having to waste time in maintaining our automated test cases, given that the system version will not suffer any changes. However, its main drawback is that we could find a design or architectural problem late in the developing process that will require a major effort to reconstruct the system.

The other way to approach a performance project is starting with the tests alongside the development stage, while the main functionalities are being released. This has the advantage of being a proper time to discover any design, architectural or even functional errors, implying less effort in solving them. Nevertheless, the cost of maintaining the performance test cases can grow considerably, given that they need to be adapted every time a change in development takes place. When maintaining the test cases, the use of GXtest as an automation tool is fundamental, due to the fact that it allows us to quickly adapt our automated test cases to the changes in the system.

OpenSTA

OpenSTA is an open source tool for performance testing and it is used for automating test cases, generating scripts that contain the http (or https) requests executed in each test case, and it allows modifications in order to perform some actions (such as introducing validations, conditions, parameterizations, etc.). In the figure below, we can appreciate the format of an OpenSTA script.

Moreover, OpenSTA allows us to mount test scenarios where we can define the amount of active users, the number of iterations, the way in which users access the system and other options that let us represent the most realistic scenario. The figure below shows an example of a scenario.

Regarding performance, OpenSTA is one of the best tool due to the following:

• It allows to introduce a large number of virtual users for each load generating computer.
• It simplifies the load distribution.
• It presents a model to assembly the scenario simulation which is well-oriented to web applications.
• It presents lots of options to analyze the results such as: numerous charts, tables, etc.
• The development productivity is much higher once you have experience with the tool.

On the other hand, a drawback of OpenSTA is the test case recording process. When recording the test cases we get a script with the http requests that were made, therefore, for someone that has no knowledge of the application, it will be really difficult to identify each step, or what each request does. Many times, in order to have a clearer script, we can introduce comments between each step during the recording, as well as timers that control the timing of each step. If on top of that, we want to add validations or parameterized variables, we can conclude that doing a script with OpenSTA is a tedious task and it may be annoying in those projects where the application’s code is being constantly changed, which means that the scripts have to be often re-generated.

It is advisable for a performance project that OpenSTA measurements are complemented with GX monitoring tools, such as JMX.

GXtest2OpenSTA

As it was mentioned in the introduction, GXtest presents an alternative that allows us to overcome this difficulty.

Thanks to its new functionality, GXtest can generate OpenSTA scripts from a previously recorded test case. This gives many advantages to the tester:

• Recording the test cases with GXtest, which is done in a much more practical way and allows to add validations and more options which are mentioned in other articles.
• Automatization of the most tedious parts of the script, without the need of wasting time in executing repetitive tasks.
• In the event of a change in the application, the possibility of taking those changes to GXtest and then re-generate the scripts in an easy way, without the need of recording them again in GXtest.
• No need of introducing validations manually, timers or parametrizations. Normally, this turns into a considerably long task and usually presents problems. GXtest automatically introduces all these elements.
• Having the secondary requests (css, gif, jpg, js, etc.) in separate files, away from the primary ones. This allows to have a clearer script which makes analysing easier.
• The use of debugging flags that condition the secondary requests and log messages.
• Support of NTLM authentication.
• Automatical management of all redirects.

How does the script generator work?

When sending the request to create an OpenSTA script, the test case will appear executing as usual in the user’s screen. Nevertheless, there are many different actions happening internally: in the course of the test case execution, a xml file is created with all the GXtest commands that are being used in the test, while http requests are recorded into a saz extension file. This files are left inside a folder with the name of the test case, where the GXtest Designer is installed, normally in: Abstracta\GXtest Designer\Performance\ExecutionLog\.

The saz format belongs to the Fiddler tool, which is a proxy that records http traffic between Internet and the PC, that then can be analyzed by introducing breakpoints, editing requests at protocol level, etc.

With these two files (xml and saz), the OpenSTA script is generated using GXtest commands from the xml file and http(s) requests from the saz file. Once the execution is completed, the resulting script will have each GXtest command that was executed, associated with the http(s) requests that are involved in each of them, with their appropriate timers, and in the case they have been introduced, the corresponding validations. Moreover, if the GXtest test case uses a variable which value is taken from a datapool, it will appear parameterized in the OpenSTA script. These files are created in the folder Abstracta\GXtest Designer\Repository.

In general, the resulting scripts are ready to be executed without doing any changes. Although, depending on the system, some modifications might be done to add conditions, variables, validations, etc., so the script will behave as real as possible.

Enable GXtest2OpenSTA

In order to generate the required files for the performance tests using OpenSTA, it is necessary to enable the generation in the menu Options -> Local Settings.

There we can enable or disable the file generation, and also configure the location of the Fiddler tool and the OpenSTA Modeller.

Summary

GXtest allows to create scripts to use in performance tests with the OpenSTA tool. When generating the scripts from GXtest, the tester saves loads of time from doing repetitive tasks, and it gives him the possibility to focus on other activities to reduce the amount of performance errors and therefore obtain a better quality product.

References

• Scott Barber; Performance Testing Guidance for Web Applications
• Scott Barber; User experience, not metrics
• Blog del CES