All Collections
Hexawise and Automated Testing
Getting Started with Hexawise Automate
Getting Started with Hexawise Automate

A reference guide for using Hexawise Automate to increase the efficiency of BDD script creation and maintenance.

Scott Johnson avatar
Written by Scott Johnson
Updated over a week ago

Hexawise Automate is a tool that makes it easy to write behavioral driven development (BDD) automated acceptance tests in Gherkin format as data-driven scripts. These data-driven test scripts are based on a Hexawise test model, so they are more maintainable than traditional BDD/Gherkin test scripts.

Automated test scripts are notoriously expensive to maintain, and BDD/Gherkin test scripts are not immune to this problem. Normally changes to the system under test, or just changes to how the system will be tested means having to make changes to tens, hundreds, or (in some cases) thousands of feature files. With data-driven Hexawise Automate test scripts though, that's not the case. Adding or removing parameter values, requirements, or constraints to the Hexawise test model updates the data driving the Hexawise Automate test scripts, but there is no need to update the test scripts themselves.

In addition, Hexawise Automate makes it easier to write correct Gherkin scripts through its syntax highlighting and autocomplete features.

Automate Usage Tips

Access Hexawise Automate by creating or opening a test model and navigating to the "Auto-Scripts" tab. Once there, select the "Automated Testing" toggle in the window header.

Dismiss the section at the top of the page containing useful hints for inserting Hexawise parameters into a Scenario or Scenario Outline, restricting a Scenario or Scenario Outline to test cases that contain certain values, etc. by clicking the Usage button.

Bring back these tips again by click the Usage button on the right side of screen, above the Preview section.

Gherkin Feature File

The Hexawise Automate screen is a smart Gherkin feature file editor. It is aware of both the parameters, parameter values, & generated test cases of the current Hexawise model and the Gherkin test script syntax. It makes it easy to create a feature file with data-driven Scenario and Scenario Outline sections that are populated by the generated test cases of the Hexawise model.

Feature

Each Gherkin script has a Feature as its first keyword followed by : and a little bit of text to describe the feature, usually in the form of a user story which is demonstrated in the placeholder text. This section does not support parameterization of any kind.

Background

A Background section is useful when a set of initial setup steps is repeated for every Scenario and Scenario Outline in the feature file. The steps of the Background are run before each Scenario Outline and Scenario. Starting a new line in Background and further sections brings up a suggestion dialog with standard Gherkin keywords.

Keep in mind that restricting a parameter in the Background (with {}, as explained below) is valid syntax and is used to apply only the selected value to all Scenarios or Scenario Outlines for a given Feature.

Scenario

Next in the feature file is the enumeration of different testing scenarios. One way to describe a test case is with a Scenario block. It contains sequential steps, expressed as plain English Given/When/Then statements, that describe the necessary actions for executing the automated test script. If the Hexawise test model generates 28 test cases, then 28 Scenario sections would need to be written out in order to achieve the combinatorial coverage.

However, instead of writing out all 28 test cases individually, with Hexawise Automate a single data-driven Scenario that includes the <Parameter Name> syntax can generate all 28 needed Scenario sections in the exported feature file. You can check the number of associated test cases in the Preview panel.

Hexawise Automate provides a way to include the value of any test case parameter in a Scenario using the <Parameter Name> syntax. Simply typing < will populate a list of autocomplete suggestions related to all of the parameters in the model, also adding:

  • Test Number (sequential number within the Scenario),

  • Test Case (constant number of the row in the complete table on the Test Cases screen)

  • Expected Outcome (from the Requirements screen, if applicable)

Wherever the <Parameter Name> appears in the test steps it will be replaced by the specified parameter's value for the test case being tested. This is conceptually very similar to how manual testing auto-scripts work in Hexawise.

For example, writing the test step:

And they select <Color> for the car's color

Means that for each generated Scenario section (one for each test case), the <Color> portion of the test step will be replaced with the value of the Color parameter in the generated test case:

And they select Deep Blue Metallic for the car's color

Writing a data-driven Scenario with at least one <Parameter> reference for each of the inputs in the 2-way test set for this model would lead to a .feature file export containing 49 Scenario sections in the feature file, one for each of the 49 Hexawise generated test cases.

A Hexawise Automate feature file can contain as many data-driven Scenario sections as needed to automate all the testing ideas contained in the Hexawise test model.

Note: it is common to put dynamic elements into quotes (e.g., And they select "<Color>" for the car's color) to make them more visually distinguishable).

Scenario Outline

A Scenario Outline behaves almost exactly like a data-driven Scenario. When using a Scenario Outline, instead of having 49 Scenario sections in the export (one for each test case), the export has one data-driven Scenario Outline with a Gherkin Examples data table attached to it that contains 49 rows (one for each test case). The Preview panel shows the first 10 rows of the Examples data table for each Scenario Outline. The order of parameters in the Examples is dictated by the script, not by the Inputs tab of Hexawise.

A Hexawise Automate feature file can contain as many Scenario Outline sections as needed to automate all the testing ideas contained in the Hexawise test model and can include both Scenario and Scenario Outline sections in the same feature file.

Where possible, using data-driven Scenario Outline sections is preferable to generating many data-driven Scenario sections as it is a more compact and direct expression of the testing ideas of the Hexawise test model.

Restricting a Scenario or Scenario Outline to Test Cases with Specific Values

Often times applications may have more or fewer test steps that need to be executed for specific test cases.

For example, in Tesla's car configurator, if the user is configuring a new Tesla Model S, there is no option for the Interior Layout. The Hexawise test model for the Tesla car configurator contains a bi-directional married pair constraint tying Model = Model S and Interior Layout = N/A together in the model shown in the example screenshots throughout this guide.

Rather than have Scenario or Scenario Outline sections that for some test cases will read:

And they select the N/A Interior Layout

Hexawise Automate provides a way to restrict a Scenario or Scenario Outline to test cases that contain a specific value using the {Parameter Name[Parameter Value]} syntax. Simply typing { will populate a list of autocomplete suggestions related to all of the parameter values in the model.

Including {Parameter Name[Parameter Value]} anywhere in the set of steps in a Scenario or Scenario Outline will restrict the test cases included in the Examples data table or in the generated Scenario sections to those that contain the specified parameter value of the specified parameter.

For example, specifying {Model[Model S]} in the following Scenario Outline restricts the generated Example data table to only those test cases that include Model = Model S.

Since this scenario is only associated with test cases including a Model S, the steps specific to a Model X (i.e. the steps mentioning parameters whose value will always be N/A given Model = Model S) can safely be left out.

Stopping here is not sufficient. The exported .feature file will not include any test cases mentioning a Model X. Creating another Scenario Outline using the syntax {Model[Model X]}, and including the additional test steps specific to the Model X will finish this testing idea.

Restricting a Feature File (all Scenarios and Scenario Outlines) to Test Cases with Specific Values

Hexawise Automate can restrict every Scenario and Scenario Outline in the feature file by using the {Parameter[Parameter Value]} syntax in the Background section.

Including Expected Outcomes from the Hexawise Forced Interactions

Often times a test case that has been forced as a combination of three or more values on the requirements screen will have an associated expected outcome that needs to be verified. Include the value of the Expected Outcome in the test script as if it were a parameter by including <Expected Outcome> in one of the steps.

Add Additional Testing Ideas Not Expressed in the Hexawise Model

There are situations where certain testing ideas did not make it into your Hexawise test model. Often these ideas represent expected outcomes or validation steps. These ideas may be necessary to complete the test script.

By using the <Parameter Name> syntax, but intentionally not using the name of a parameter in the Hexawise test model, you can include <Some Additional Idea> and Automate will add an additional column to the Gherkin Examples data table. This column will be empty in the exported feature file, allowing you to fill in the necessary blanks.

In the example below, the Hexawise test model does not have a parameter called Browser. but <Browser> has been included in one of the steps for a Scenario Outline.

If there is a parameter in your model that contained only a single value (because that value must be true for all of the test cases generated), you can remove that parameter from the model entirely and still have an equally-detailed Automate script. For example, all instances of Tesla Model S and Model X must have the Premium Upgrade Package = Equipped.

To include this idea there are two choices:

  1. Simply type out the needed step without any reference to a <Parameter Name> or {[Parameter Name]Parameter Value}. This is most appropriate if it is assumed that this testing idea is unlikely to expand to other variations in the future.

  2. Include a parameter in your model called Premium Upgrade Package with a single value Equipped and insert <Premium Upgrade Package> in your Automate script. This is most appropriate if it is expected this testing idea will expand to other variations in the near future.


Including One-off and Negative Testing Ideas

Take advantage of the free-form nature of a Gherkin Scenario to include testing ideas that don't come from the Hexawise generated test cases by adding Scenario sections that don't include any <Parameter Name> or {Parameter Name[Parameter Value]} syntax.

Using a Scenario also makes it easy to include negative test cases without making adjustments to your model.

Creating and Exporting Multiple Feature Files

Sometimes having separate feature files is beneficial for test management & execution purposes. You can now achieve that in Hexawise Automate by clicking the blue “+” icon next to the name of the default feature section. You will need to provide the name for the new tab, all names should be unique within the same Hexawise model.

When you export into “Gherkin Feature File” or “Automated Test Framework”, you will have an option to select all scripts or only the specific ones.

Please make sure that you cover all the generated test cases across tabs, especially if you leverage the value restrictions.

Did this answer your question?