Difference between revisions of "GXtest Generator User Manual"

Latest revision as of 15:48, 26 February 2017

GXtest Generator is a tool designed to help us with the startup of a new test project in GXtest, in an automated fashion. GXtest Generator is a GeneXus extension, and can be installed along with GXtest or as a standalone tool. Using this extension, we can automatically generate test cases and test data with only one click.

Before we begin, we must have an installed and authorized GXtest copy somewhere in our local network (may be on anothe computer), where the test cases will be run (See GXtest Installation guide).

Automated test case generation

This tool takes advantage of GeneXus abstraction levels trough transactions, webpanels and work with patterns information (available in the Knowledge Base), to generate GXtest test cases and datapools in an automated fashion.

Which functionalities are tested by the automatically generated test cases?

For each transaction in the Knowledge Base, a test case is generated to test the CRUD operations over the entity represented by the transaction: Create (insert), Retrieve (search), Update, and Delete. For example, if we have the Client transaction in the KB, GXtest Generator will create a test case that inserts a new Client, then it searchs the new client in the "Work With" page, then updates some data about the client, and finally it will delete de created Client.

For each webpanel, a test case that perform actions on all the webpanel controls is generated

How good or complex are the generated test cases?

Although the test cases are created to test the CRUD operations, the main purpose of this test cases is that the user can start with automated testing quickly, and not start from scratch.

Also, these test cases can be edited, improved, or reused inside other more complex test cases.

How can I see the generated test cases?

After the generation project is completed, all test cases and datapools are imported into a GXtest project named the same as the KB, and the suffix "_Auto".

Starting GXtest Generator in GeneXus

In GeneXus IDE, we can open GXtest Generator from the "GXtest" menu, by selecting the "Autogenerate TestCases" option.

In the window that opens up, we can see the existing project (if we already have one for this KB), or the button that allows us to start the generation process: "Generate tests".

If we need to, we may configure some aditional options in the extension configuration page, before we start the process.

Configuration Options

Inside GeneXus menu, in Tools -> Options, exists a section where all GXtest extensions are configured.

In section General you can specify:
- Data Dictionary Path: The file path where dictionary data is loaded. An example is available here
- Include Unreferences Transactions: Sets whether all transactions and objects may be generated (True), or only transactions or pages called from any GeneXus Main object (False).
- Language: The lenguage in which the application is generated and tested. This is required in order to check the GeneXus automatic validation messages.
- Startup Test Case: an initial test case that will be included at the start of every generated test case. This becomes handy when you need to execute a step prior to the execution of all test cases, for example a login test case.
- Test depth: Sets the test depth of the generated test cases. See additional info in GXtest Generator - Test depth.
- Visit Web Pages: When set to True, it generates an additional test cases which visits every web page of the application. This becames usefull in order to check that all pages are correctly deployed in the test environment.

Manager section
- Here you may configure the GXtest Manager schedule information about the generated test cases, in order to start unattended execution of the project tests.

Connection section
- All the GXtest database connection-related data is configured, where the generated test cases and project will be saved.

Advanced section:
- Temp XML path: the temporary folder where the generated test cases are stored, before they get imported into GXtest project. Also, this path is the location where the generation log file is saved.
- Update KB: It sets the update policy applied if the KB already exists in GXtest. If the KB must be always updated then you should choose Always Update. To update the KB objects only when no conflicts are raised (new objects only) select Update if no conflicts. Choose Never Update if you do not want to impact the KB changes into GXtest.

GXtest Generator Execution

While executing the generation process, details about completed steps will be shown in the grid, and also in the GeneXus output window. All analyzed objects, generated test cases and datapools will be displayed in the output.

Updating the generated test cases when the KB changes

GXtest Generator main purpose is to be executed one time only, in order to initialize and create the test project, to begin working with GXtest Designer, creating new test cases and improving the automatically generated ones.

However, some logic exists in the generator in order to allow us to automatically create new test cases when the KB has new objects, and we have already generated the test project in the past. To accomplish, that the user must set the name of the new project to the same name of the existing project.

For each "Generate project" execution, every test case and test data is generated again. If the target project already exists, the new test cases will be imported, and the existing test cases and datapools will be overwritten with the new version of each, except when the following conditions are met:

The test case was manually updated in GXtest Designer, and the Update Automatically option is disabled (it becomes disabled automatically whenever we edit the automatically generated test case). This option is available in the test case properties window.

The Datapool was manually updated in GXtest Designer (its columns or data), and the Update Automatically option is disabled (it becomes disabled automatically whenever we edit the automatically generated datapool or its data). This option is available in the datapool edition window.

Dictionaries

The data dictionaries can be loaded by specifying an XML file path in the configuration options, and this kind of file should look like the example below.

<?xml version="1.0" encoding="utf-8" ?>
<Library version="1.0">
  <Catalogue name="Countries" defaultDomain="">
    <DataRow>Italy</DataRow>
    <DataRow>Uruguay</DataRow>
    <DataRow>Canada</DataRow>
    <DataRow>Japan</DataRow>
    <DataRow>Brazil</DataRow>
  </Catalogue>
  <Catalogue name="Mails" defaultDomain="EMail">
    <DataRow>guest@mailx.com</DataRow>
    <DataRow>coconut@mailx.com</DataRow>
    <DataRow>fashionmail@mailx.com</DataRow>
    <DataRow>abstracta@mailx.com</DataRow>
  </Catalogue>
  <Catalogue name="GivenNames" defaultDomain="">
    <DataRow>Juan</DataRow>
    <DataRow>Miguel</DataRow>
    <DataRow>Peter</DataRow>
    <DataRow>John</DataRow>
    <DataRow>Jeremy</DataRow>
  </Catalogue>
  <Catalogue name="Products" defaultDomain="">
    <DataRow>Smartphone</DataRow>
    <DataRow>Running Shoes</DataRow>
    <DataRow>Laptop</DataRow>
    <DataRow>Teddy Bear</DataRow>
    <DataRow>Barbie</DataRow>
  </Catalogue>
  <Catalogue name="Categories" defaultDomain="">
    <DataRow>Movies</DataRow>
    <DataRow>Sports</DataRow>
    <DataRow>Office</DataRow>
    <DataRow>Toys</DataRow>
  </Catalogue>
</Library>

The data Dictionary or catalog can be associated with a KB attribute, so the data included in the test is taken from the dictionary, instead of generating it randomly. This gives better semantics to the data used in the test cases, so the data is easier to understand.

For example, given the dictionary of the example, and the transaction Client with the following structure:

If we want the Generator to take a given name from the "GivenNames" catalog when filling the data for the ClientFirstName attribute (for example inside the Client test cases), then we must look into the ClientFirstName attribute properties, and set the value "GivenNames" on the "Data Dictionary" property combo.

The values that appears in the combo are the same catalogs loaded from the dictionaries XML file.

Another way of using dictionaries values for the test case Generator, is done by matching a catalog with a GeneXus Domain. We can use as an example the ClientEmail attribute, which has the Email domain applied to it. If we look at the dictionaries file, we have a specific catalog with some Email addresses, which is associated with the Email domain through the defaultDomain attribute.

...
  <Catalogue name ="Mails" defaultDomain="EMail">
    <DataRow>guest@mailx.com</DataRow>
    <DataRow>coconut@mailx.com</DataRow>
...

By applying both configurations (on ClientFirstName and ClientEmail), GXtest Generator will automatically fill the Client Datapool with values like the following:

Troubleshooting GXtest Generator

When executing the project generation process on GeneXus Ev2 Upgrade 2 or below, the following error may be shown:

   ========== Generation started ==========
   Error: Método no encontrado: 'Artech.Architecture.Common.Objects.KBEnvironment Artech.Architecture.Common.Objects.KBModel.get_Environment()'.

The problem is fixed by installing the latest GeneXus upgrade available for Evolution 2.

@@ Line 1: / Line 1: @@
 [[category:GXtest Guides]]
-{{Idiomas|Manual de Usuario de GXtest Generator|GXtest Generator User Manual|GXtest Generator User Manual}}
+{{Idiomas|Manual de Usuario de GXtest Generator|GXtest Generator User Manual|GXtest Generator ユーザーズマニュアル}}
-GXtest Generator es una herramienta que nos ayuda a comenzar a automatizar pruebas con GXtest, a partir de la generación automática de casos de prueba. La extensión GXtest Generator que se instala en GeneXus.
-Antes de comenzar se debe tener instalado y activado GXtest en algún PC, de modo de poder ejecutar las pruebas (Ver [[Installation guide]] de GXtest).
+GXtest Generator is a tool designed to help us with the startup of a new test project in GXtest, in an automated fashion. GXtest Generator is a GeneXus extension, and can be installed along with GXtest or as a standalone tool. Using this extension, we can automatically generate test cases and test data with only one click.
-== Generación Automática de Casos de Prueba ==
+Before we begin, we must have an installed and authorized GXtest copy somewhere in our local network (may be on anothe computer), where the test cases will be run (See GXtest [[Installation guide]]).
-Gracias a la abstracción que provee GeneXus a través de las transacciones y patrones, se pueden generar casos de prueba para GXtest de manera automática, a partir de la información que existe de dichos objetos en la base de conocimiento.
+== Automated test case generation ==
-* ¿Qué testean los casos de prueba automáticos?
+This tool takes advantage of GeneXus abstraction levels trough transactions, webpanels and work with patterns information (available in the Knowledge Base), to generate GXtest test cases and datapools in an automated fashion.
-Se genera un caso de prueba para cada entidad de la aplicación, el cual probará el Alta, Baja, Modificación, Visualización y Búsqueda de una instancia de dicha entidad. Por ejemplo, para la transacción Client, se dará de alta un cliente, se lo buscará en el "Trabajar Con" asociado, se lo modificará, y finalmente se lo dará de baja.
-* ¿Para qué me sirven estos casos de prueba?
-Si bien sirven para probar el ABM (Alta, Baja y Modificación) de todas las entidades de la KB, la principal finalidad de dichos casos de prueba es utilizarlos como base para comenzar la automatización de pruebas sobre una aplicación.
-Dependiendo del tipo de aplicación, los ABM de datos pueden significar un 80% de la totalidad de las funcionalidades de la aplicación. Por ello, generar estas pruebas de forma automática implica reducir los tiempos de automatización de gran forma, pudiendo utilizar ese tiempo en extender las pruebas generadas, o realizar nuevos casos de prueba, utilizando incluso los ya generados.
-* ¿Cómo genera GXtest los casos de prueba?
+* Which functionalities are tested by the automatically generated test cases?
-** Se examinan todas las transacciones y por cada una se determina si es posible generar el caso de prueba, y el flujo que tendrá el ABM dependiendo si tiene aplicado algún pattern WorkWith.
+For each transaction in the Knowledge Base, a test case is generated to test the CRUD operations over the entity represented by the transaction: Create (insert), Retrieve (search), Update, and Delete. For example, if we have the Client transaction in the KB, GXtest Generator will create a test case that inserts a new Client, then it searchs the new client in the "Work With" page, then updates some data about the client, and finally it will delete de created Client.
-** Durante la generación de cada objeto, se analizan todos sus atributos, y para cada uno se determinan si es necesario generar datos (si son editables, no son calculados, etc.), y cómo se van a generar los mismos. Algunas de las estrategias son:
-*** Si es una FK, se extraen de la base de datos de la aplicación, consultando la tabla asociada a la FK, incluyendo valores para atributos inferidos por éste (por ejemplo, si en la transacción está como FK el ClienteId,pero en realidad en la pantalla se ingresa el ClienteNombre).
-*** Si el control tiene valores predefinidos (por ejemplo con combobox con opciones estáticas) se toma uno aleatorio.
-*** En otro caso, se genera de forma aleatoria un valor asociado al tipo del atributo, y que cumpla con las máscaras definidas en el control o atributo.
-* ¿Cómo hace GXtest para obtener los datos de la base de datos de la aplicación?
+For each webpanel, a test case that perform actions on all the webpanel controls is generated
-Para la consulta de datos a la BD, se aprovecha la abstracción que provee GeneXus para acceder a ésta sin tener que configurar un conector. Esto se logra generando temporalmente en la KB un procedure y un data provider que realizan la consulta sobre cierta tabla, y lo exponen como un servicio REST.
-* ¿Cómo accedo a los casos de prueba generados?
+* How good or complex are the generated test cases?
-Después de generados los casos de prueba y los datapools, se importan de forma automática dentro de GXtest, en un nuevo proyecto que lleva el nombre de la KB y el sufijo "_Auto".
+Although the test cases are created to test the CRUD operations, the main purpose of this test cases is that the user can start with automated testing quickly, and not start from scratch.
+Also, these test cases can be edited, improved, or reused inside other more complex test cases.
-== GXtest Generator ==
+* How can I see the generated test cases?
-La herramienta se instala como una extensión de GeneXus Ev1 o Ev2. De esta forma, las funcionalidades estarán accesibles integradas al entorno de desarrollo.
+After the generation project is completed, all test cases and datapools are imported into a GXtest project named the same as the KB, and the suffix "_Auto".
-=== Instalación de la Extensión en GeneXus ===
-La funcionalidad se distribuye en un instalador muy simple.
-Durante la instalación:
-* Se deberá indicar el path de la instalación de GeneXus donde se quiere instalar la extensión. Puede aplicarla en más de una instalación de GeneXus (si tuviera varias).
-* Se deberá proporcionar una base de datos de GXtest, donde la extensión guardará los casos de prueba generados. La versión de GXtest instalada debe ser 1.2.0 +, y además debe coincidir con la versión de la extensión.
-Las versiones soportadas para la base de datos son Microsoft SQLServer 2005 o superior, pudiendo ser una Express Edition.
+=== Starting GXtest Generator in GeneXus ===
+In GeneXus IDE, we can open GXtest Generator from the "GXtest" menu, by selecting the "Autogenerate TestCases" option.
+In the window that opens up, we can see the existing project (if we already have one for this KB), or the button that allows us to start the generation process: "Generate tests".
-=== Acceso a la GXtest Generator en GeneXus ===
+If we need to, we may configure some aditional options in the extension configuration page, before we start the process.
-Al abrir GeneXus, aparecerá un nuevo menú "GXtest" que tendrá el ítem "Autogenerate TestCases".
-Dicha opción de menú abre la ventana de generación automática de GXtest Generator.
-Allí se verá información de testing asociada a la KB que está abierta (si es que ya generamos un proyecto en GXtest), y estará disponible el botón para iniciar el proceso: "Generate tests".
-En caso de ser necesario, antes de comenzar a generar los casos de prueba es posible configurar las opciones de generación que la extensión provee.
-=== Opciones de Configuración ===
+=== Configuration Options ===
-En el menú Tools->Options en GeneXus, hay una sección específica para configurar la extensión de GXtest.
+Inside GeneXus menu, in Tools -> Options, exists a section where all GXtest extensions are configured.
-* En la sección general, se puede configurar:
+* In section General you can specify:
-** Data Dictionary Path: El path a un diccionario de datos en formato XML. Este diccionario se puede utilizar para atributos que tengan una semántica bien definida, por ejemplo nombres, o direcciones de email. Esos diccionarios se pueden asociar a cualquier atributo o dominio. Ver [[GXtest Generator - Dictionaries]].
+** Data Dictionary Path: The file path where dictionary data is loaded. An example is available [[GXtest Generator User Manual#Dictionaries | here]]
-** Include Unreferences Transactions: Indica si se debe incluir a todas las transacciones y objetos (True), o solamente a transacciones y objetos que sean alcanzables por algún Main (False).
+** Include Unreferences Transactions: Sets whether all transactions and objects may be generated (True), or only transactions or pages called from any GeneXus Main object (False).
-** Language: El lenguaje en el que se probará la aplicación. Esto es requerido para predefinir el lenguaje de los mensajes que GXtest validará.
+** Language: The lenguage in which the application is generated and tested. This is required in order to check the GeneXus automatic validation messages.
-** Startup Test Case: un caso de prueba inicial que se ejecutará antes de cada TestCase. Ver [[GXtest Generator - Include Login Test Case]].
+** Startup Test Case: an initial test case that will be included at the start of every generated test case. This becomes handy when you need to execute a step prior to the execution of all test cases, for example a login test case.
-** Test depth: Indica la profundidad que tendrán los casos generados, en lo referido a la cantidad de combinaciones de datos que se generarán para la prueba. Ver [[GXtest Generator - Test depth]]. Tiene cuatro opciones:
+** Test depth: Sets the test depth of the generated test cases. See additional info in [[GXtest Generator - Test depth]].
-*** Minimum (genera un único dato válido para cada campo).
+** Visit Web Pages: When set to True, it generates an additional test cases which visits every web page of the application. This becames usefull in order to check that all pages are correctly deployed in the test environment.
-*** Basic (genera varios registros válidos para cada campo, según casos borde de cada campo).
-*** Basic with rules(igual al Basic, pero también toma en cuenta las rules de los objetos).
-*** Valid and Invalid (datos conjuntos de datos válidos e inválidos, de forma de testear las validaciones sobre los datos inválidos).
-** Visit Web Pages: Si se configura en True, genera un caso de prueba que visita todos los webpanels encontrados en la KB, de forma de validar que los mismos estén generados y deployados correctamente en la aplicación.
-* En la sección Manager se puede configurar los datos para que GXtest Manager agende la tarea, y ordene la ejecución de los casos de pruebas generados, en el GXtest Executor configurado.
+* Manager section
+** Here you may configure the GXtest Manager schedule information about the generated test cases, in order to start unattended execution of the project tests.
-* En la sección Connection se configuran los datos de conexión a la base de datos de GXtest, donde se creará el proyecto y los casos de prueba generados.
+* Connection section
+** All the GXtest database connection-related data is configured, where the generated test cases and project will be saved.
-* En la sección Advanced se configuran opciones mas avanzadas de la generación.
+* Advanced section:
-** Temp XML path: el directorio temporal donde se almacenan los casos de prueba y datapools autogenerados. Además allí se genera un archivo de Log donde se pueden ver detalles de la generación y errores ocurridos durante el proceso.
+** Temp XML path: the temporary folder where the generated test cases are stored, before they get imported into GXtest project. Also, this path is the location where the generation log file is saved.
-** Ignore List: esta opción permite especificar una lista separada por comas de objetos que no se deben incluir en la generación de los casos de prueba. Por ejemplo: "Client,Invoice*".
+** Update KB: It sets the update policy applied if the KB already exists in GXtest. If the KB must be always updated then you should choose Always Update. To update the KB objects only when no conflicts are raised (new objects only) select Update if no conflicts. Choose Never Update if you do not want to impact the KB changes into GXtest.
-** Update KB: Indica si la KB en GXtest debe ser actualizada cada vez que se genera el proceso (Always Update), solamente cuando no hay conflictos (Update if no conflicts), o nunca (Never Update).
 === GXtest Generator Execution ===
@@ Line 85: / Line 66: @@
 While executing the generation process, details about completed steps will be shown in the grid, and also in the GeneXus output window. All analyzed objects, generated test cases and datapools will be displayed in the output.
+== Updating the generated test cases when the KB changes ==
-== Updating the generated test cases while the KB grows ==
 GXtest Generator main purpose is to be executed one time only, in order to initialize and create the test project, to begin working with GXtest Designer, creating new test cases and improving the automatically generated ones.
-However, some logic exists in the generator in order to allow us to automatically create new test cases when the KB has new objects, and we already generated the startup project.
+However, some logic exists in the generator in order to allow us to automatically create new test cases when the KB has new objects, and we have already generated the test project in the past. To accomplish, that the user must set the name of the new project to the same name of the existing project.
-For each "Create project" execution, every test case and test data is generated again. If the target project already exists, the test cases and datapools will be overwritten with the new ones, except if the following conditions are met:
+For each "Generate project" execution, every test case and test data is generated again. If the target project already exists, the new test cases will be imported, and the existing test cases and datapools will be overwritten with the new version of each, except when the following conditions are met:
-* The test case was manually updated in GXtest Designer, and if the Update Automatically option is disabled (it becomes disabled automatically whenever we edit the automatically generated test case). This option is available in the test case properties window.
+* The test case was manually updated in GXtest Designer, and the Update Automatically option is disabled (it becomes disabled automatically whenever we edit the automatically generated test case). This option is available in the test case properties window.
 [[File:TCUpdateAuto.png|center]]
-* The Datapool was manually updated in GXtest Designer (its columns or data), and if the Update Automatically option is disabled (it becomes disabled automatically whenever we edit the automatically generated datapool or its data). This option is available in the datapool properties window
+* The Datapool was manually updated in GXtest Designer (its columns or data), and the Update Automatically option is disabled (it becomes disabled automatically whenever we edit the automatically generated datapool or its data). This option is available in the datapool edition window.
+== Dictionaries ==
+The data dictionaries can be loaded by specifying an XML file path in the configuration options, and this kind of file should look like the example below.
+<pre>
+<?xml version="1.0" encoding="utf-8" ?>
+<Library version="1.0">
+  <Catalogue name="Countries" defaultDomain="">
+    <DataRow>Italy</DataRow>
+    <DataRow>Uruguay</DataRow>
+    <DataRow>Canada</DataRow>
+    <DataRow>Japan</DataRow>
+    <DataRow>Brazil</DataRow>
+  </Catalogue>
+  <Catalogue name="Mails" defaultDomain="EMail">
+    <DataRow>guest@mailx.com</DataRow>
+    <DataRow>coconut@mailx.com</DataRow>
+    <DataRow>fashionmail@mailx.com</DataRow>
+    <DataRow>abstracta@mailx.com</DataRow>
+  </Catalogue>
+  <Catalogue name="GivenNames" defaultDomain="">
+    <DataRow>Juan</DataRow>
+    <DataRow>Miguel</DataRow>
+    <DataRow>Peter</DataRow>
+    <DataRow>John</DataRow>
+    <DataRow>Jeremy</DataRow>
+  </Catalogue>
+  <Catalogue name="Products" defaultDomain="">
+    <DataRow>Smartphone</DataRow>
+    <DataRow>Running Shoes</DataRow>
+    <DataRow>Laptop</DataRow>
+    <DataRow>Teddy Bear</DataRow>
+    <DataRow>Barbie</DataRow>
+  </Catalogue>
+  <Catalogue name="Categories" defaultDomain="">
+    <DataRow>Movies</DataRow>
+    <DataRow>Sports</DataRow>
+    <DataRow>Office</DataRow>
+    <DataRow>Toys</DataRow>
+  </Catalogue>
+</Library>
+</pre>
+The data Dictionary or catalog can be associated with a KB attribute, so the data included in the test is taken from the dictionary, instead of generating it randomly. This gives better semantics to the data used in the test cases, so the data is easier to understand.
+For example, given the dictionary of the example, and the transaction Client with the following structure:
+[[Image:ClientTRN.PNG|center]]
+If we want the Generator to take a given name from the "GivenNames" catalog when filling the data for the ClientFirstName attribute (for example inside the Client test cases), then we must look into the ClientFirstName attribute properties, and set the value "GivenNames" on the "Data Dictionary" property combo.
+[[Image:AttDataDictionary.PNG|center]]
+*The values that appears in the combo are the same catalogs loaded from the dictionaries XML file.
+Another way of using dictionaries values for the test case Generator, is done by matching a catalog with a GeneXus Domain. We can use as an example the ClientEmail attribute, which has the Email domain applied to it. If we look at the dictionaries file, we have a specific catalog with some Email addresses, which is associated with the Email domain through the defaultDomain attribute.
+<pre>
+...
+  <Catalogue name ="Mails" defaultDomain="EMail">
+    <DataRow>guest@mailx.com</DataRow>
+    <DataRow>coconut@mailx.com</DataRow>
+...
+</pre>
+By applying both configurations (on ClientFirstName and ClientEmail), GXtest Generator will automatically fill the Client Datapool with values like the following:
+[[Image:DP Dictionaries.PNG|center]]
+== Troubleshooting GXtest Generator ==
+When executing the project generation process on GeneXus Ev2 Upgrade 2 or below, the following error may be shown:
+    ========== Generation started ==========
+    Error: Método no encontrado: 'Artech.Architecture.Common.Objects.KBEnvironment Artech.Architecture.Common.Objects.KBModel.get_Environment()'.
+The problem is fixed by installing the latest GeneXus upgrade available for Evolution 2.

Difference between revisions of "GXtest Generator User Manual"

Latest revision as of 15:48, 26 February 2017

Contents

Automated test case generation

Starting GXtest Generator in GeneXus

Configuration Options

GXtest Generator Execution

Updating the generated test cases when the KB changes

Dictionaries

Troubleshooting GXtest Generator

Navigation menu

Personal tools

Namespaces

Variants

Views

Actions

Search

Toolbox