Test parameterization, variables, and datasets

Test parameterization involves running the same test with multiple different values. This type of data-driven testing allows you to test the same basic logic or functionality in your application with multiple inputs, without having to copy and maintain duplicate test cases.

Within TestRail, you can run parameterized tests using test data,  a general term that encompasses the concept of variables, datasets, and values specific to each project. Test data allow testers to create test variables and datasets (containing values for each variable) to use in their test cases and test runs. Variables can be inserted into the text and string fields of test cases, and then the corresponding value is used in the test in a test run. 

By using parameterized tests, you can increase the flexibility of your test cases and allow for greater test coverage with fewer cases to manage.

The following sections cover how to utilize test data, variables, and datasets in TestRail:

• User Permissions
• Test Data Management
• Adding Test Data Variables to Test Cases
• Using Test Data in Test Runs & Plans
• Test Data API Endpoints

User permissions

Test Data is specific to each project in your TestRail instance. Administrators use roles and permissions to manage how testers can use this test data. You can create roles such as “Lead,” “Tester,” and “Guest” and apply permissions based on what actions a person in each role performs. 

The Test Data and Variables permission highlighted below affects how your user roles in each project are able to manage this data. 

• Add/Edit: This allows users to add and edit variables and datasets within the Test Data management area.
• Delete: This allows users to delete variables and/or datasets entirely.

Test data management

Each project in TestRail includes a dashboard dedicated to viewing and managing test data available for that project. You can access the dashboard from your lists of test runs, test suites, or test cases by clicking the View Test Data button.

Within the test data management area, you can add new test data values, view and edit existing data, and import or export test data via CSV. This view presents a table of your existing datasets and variables. Note that test data is unique to each project.

• A Variable is the placeholder which you insert into your test cases. You can have up to 500 variables.
• A Dataset is a collection of values corresponding to each variable. There is always one default dataset and you can add up to 99 more datasets for a total of 100 datasets by adding new columns to your test data table. 

Variable and Dataset names must meet the following criteria:

• Must contain only letters, numbers, and underscores (no spaces or special characters)
• Must not begin with an underscore
• Must not exceed 50 characters
• Variable names must be unique within the current project
• Dataset names must be unique within the current project
• Names are case sensitive

Datasets are NOT required to have a value for every variable. If a dataset’s value is blank, then any associated test cases use the value from the default dataset, if it exists.

The test data dashboard allows you to perform the following actions:

Edit your test data

Click Edit to open the edit mode for your test data. 

• To add a new variable, type a new variable name into a blank row of the table’s Variable column. 
• To add a new dataset, type a new dataset name into the header row of the table. You can then add new values to your new datasets.
• Click on the row number or column header to select the entire row or column.  To delete the entire selected row or column, click delete the values.
• You must click Save to retain your changes.

Create test runs

Click Create Test Runs to create test runs based on your data sets. This action opens the Create Test Runs dialog:

Select the dataset(s) you would like to use for your test run(s), and then click Continue. TestRail now allows you to select what you want to do with the selected dataset(s), as shown in the following dialog.

This dialog provides you with the following options for your new run(s):

• Create a test plan containing your test runs
  • This option creates one test plan that contains a test run for each of the datasets selected. TestRail then opens the view for creating a new test plan with your new test runs included.
• Add your test runs to a current test plan
  • This option creates a new test run for each of the datasets selected, but also allows you to add these test runs into an existing test plan. Choosing this option prompts you for the target test plan, and then adds these runs automatically to the selected plan.
• Create individual test runs
  • This option creates a new, separate test run for each of the datasets selected. You can select test cases for these runs, but all of the test runs are created with the same selection of test cases.

Import test data

Click Import CSV to import a CSV file of datasets and variables.

Select your file and appropriate file encoding, and then select the CSV delimiter to begin the import process.

TestRail displays a preview of the imported data. Verify that the preview looks accurate before importing the data. Any items appearing in green reflect an updated or new value.

Export your test data

Click Export CSV to export your datasets to a CSV file. In the Export CSV dialog, select the dataset(s) you would like to export, and then click Export.

Adding test data variables to test cases

When you have datasets and variables inside of your project, you can then use this data inside of your test cases. When editing a test case, you can insert a variable into any markdown-enabled text, string, and separate step field, including shared test steps.

TestRail does not support using variables in any fields not mentioned above, in test case titles, or any in fields outside of test cases (Section names, Test Plan descriptions, etc.).

Insert a Variable into a test case

When editing a test case, you can quickly insert a variable name by typing the name directly into the field, surrounded by percent symbols ‘%’, or by using the variable button to select an existing variable.

Using test data in test runs & plans

When a test run utilizes a dataset and test cases in the test run call variables, TestRail automatically uses the associated dataset values inside of the run’s tests. These values are highlighted in your test, allowing you to quickly identify which variables are in use:

You can configure Test Runs to utilize datasets by editing the test run and selecting a dataset from the drop-down option.

You also can configure Test Runs inside of Test Plans to utilize datasets by clicking Description, References & Data, and then clicking the drop-down menu to choose the dataset, as shown in the following image:

Test data API endpoints

TestRail’s Test Data feature is also supported by the TestRail API. The following API endpoints are available to view and manage your test data:

• get_dataset
• get_datasets
• add_dataset
• update_dataset
• delete_dataset
• get_variables
• add_variable
• update_variable
• delete_variable

Test Run and Test Plan endpoints also support dataset selection and usage in test runs.

Please review our API documentation for further details.