Parameter priorities
User can choose to set parameters from different places like default config file, environment variables, custom config file, cli parameters or in some cases use default values. The priority of setting parameters from different places is as per the table below, where 1 is the highest and 5 is the lowest.
| Priority | Source |
|---|---|
| 1 | cli parameters |
| 2 | custom config file |
| 3 | environment variables |
| 4 | default config file |
| 5 | default value |
Configuration files
Configuration files can be used to pass parameters, options, settings and preferences to the TestRail CLI. The configuration files should be written in YAML format.
Possible fields:
| Field name | Description |
|---|---|
| host | Specifies the URL of the TestRail instance in which to send the results |
| project | Specifies the name of the Project the Test Run should be created under |
| project_id | Project id. Will be only used in case project name will be duplicated in TestRail |
| username | Username for authentication |
| password | Password for authentication |
| key | API key for authentication |
| file | Specifies the filename and/or path of the result file to be used |
| title | Specifies the title of the Test Run to be created in TestRail |
| verbose | Enables verbose mode when true (false by default) |
| verify | Verify the data was added correctly |
| insecure | Allow insecure requests |
| silent | Enables silence mode (only stdout) when true (false by default) |
| config | Specifies the filename and/or path of the configuration file to be used |
| batch_size | Specifies the batch size of results to pass to TestRail |
| timeout | Specifies how many seconds to wait for more results before termination |
| auto_creation_response | Sets the response for auto creation prompts. If not set user will be prompted whether to create resources (suite, test case etc.) or not. |
| suite_id | Specifies the Suite ID for the Test Run to be created under |
| run_id | Specifies the Run ID to be updated |
| milestone_id | Specifies the Milestone ID for the Test Run to be created under |
| close_run | Specifies whether to close the run after adding all the results (false by default) |
| case_fields | Dictionary with case fields to be filled on case creation as a key value pair |
| result_fields | Dictionary with result fields to be filled on results creation as a key value pair |
| run_description | Text to be added to the run description (for example, if you want to add the link to your CI job) |
Configuration file example:
host: https://fakename.testrail.io/
project: My TestRail Project
username: myuser@name.com
password: StrongP@ssword
file: PATH/result_file.xml
title: Automated Tests
config: PATH/alternate_config.yml
batch_size: 20
timeout: 5.5
auto_creation_response: true
case_fields:
type_id: 1,
priority_id: 3
custom_field_id: [2, 4]Default configuration file
Default configuration file should be named config.yaml or config.yml and be stored in the same directory as the trcli executable file. The default path for pip installation of executable depends on your system and python settings (venv).
Custom configuration file
Apart from default configuration file, a custom one can be passed after -c/--config as a parameter.
Environment variables
It is possible to pass parameters and options to the trcli tool by setting environment variables. The variables should be named as follows: TR_CLI_PARAMETER_NAME_CAPITALIZED
For example, for -h/--host: TR_CLI_HOST
The --yes/--noparameters are a special case. Use the variable name TR_CLI_AUTO_CREATION_RESPONSE with values true to agree for auto creation or false to disagree.
Note that in case there is a - in the parameter name it should be changed to _.
Example: For --project-id, the environment variable name will be TR_CLI_PROJECT_ID.
About setting environment variables
There are different ways of setting variables depending on system used. Below are some examples of how to set an environment variable for a single session, meaning it will not be persisted across different terminal sessions.
# Windows cmd
set TR_CLI_PROJECT=project name
# Windows powershell
$env:TR_CLI_PROJECT="project name"
# Linux
export TR_CLI_PROJECT="project name" for Linux-like systems