Centralizing test results in TestRail is a very compelling use case. It helps you consolidate all your testing efforts in one place, allowing you to have more visibility and control over testing, as well as enabling you to generate any of the rich test reports provided by TestRail.
With the ability to programmatically create test runs and import results, you can implement integrations such as:
- Create runs and upload test results from test automation suites
- Updating test runs with outsourced test results
| Real examples | Use case |
|---|---|
| TestRail CLI | Test run creation and results upload from test automation reports |
| Functionize | Test run creation and results upload from automated tests |
| Katalon | Test run creation and results upload from automated tests |
| Global App Testing | Submits results added by external testers |
Creating a test run
Test runs work as containers for the tests you want to execute, so you can submit your results. To add a simple test run, you can use the following endpoint.
POST index.php?/api/v2/add_run/{project_id}
curl -X POST \
-u "user@example.com:password" \
-F "@path/to/data.json" \
"https://example.testrail.io/index.php?/api/v2/add_run/12"
{
"suite_id": 1,
"name": "New test run",
"description": "Test run description",
"assignedto_id": 5,
"include_all": false,
"case_ids": [4076, 4078, 4079]
}{
"id": 228,
"suite_id": 53,
"name": "New test run",
"description": "Test run description",
"milestone_id": null,
"assignedto_id": 5,
"include_all": false,
"is_completed": false,
"completed_on": null,
"config": null,
"config_ids": [],
"passed_count": 0,
"blocked_count": 0,
"untested_count": 3,
"retest_count": 0,
"failed_count": 0,
"custom_status1_count": 0,
"custom_status2_count": 0,
"custom_status3_count": 0,
"custom_status4_count": 0,
"custom_status5_count": 0,
"custom_status6_count": 0,
"custom_status7_count": 0,
"project_id": 27,
"plan_id": null,
"created_on": 1684936873,
"updated_on": 1684936873,
"refs": null,
"dataset_id": null,
"created_by": 18,
"url": "https://marketing80testing.testrail-staging.com/index.php?/runs/view/228"
}Adding results to a test run using case IDs
To add results to a test run, you can make use of case IDs to map each result against the respective test case. Besides actual results, you can also add a simple comment, as you can see in the example body content. To add your results, you can use the following endpoint.
If you configured custom result fields, you can also submit values using the pattern custom_field_name, such as custom_failure_severity in the example below.
You can find the status ids and their details by executing the following request:
GET index.php?/api/v2/get_statuses
The default statuses are:
1 - Passed
2 - Blocked
3 - Untested (default value, can’t be used to submit a new status)
4 - Retest
5 - Failed
POST index.php?/api/v2/add_results_for_cases/{run_id}
curl -X POST \
-u "user@example.com:password" \
-F "@path/to/data.json" \
"https://example.testrail.io/index.php?/api/v2/add_results_for_cases/228"
{
"results": [
{
"case_id": 4076,
"status_id": 1,
"comment": "This test passed",
"elapsed": "5m",
"environment": "qa03"
},
{
"case_id": 4078,
"status_id": 5,
"comment": "This test failed",
"assignedto_id": 5,
"elapsed": "3m",
"environment": "qa03",
"defects": "TR-7",
"custom_failure_severity": 3
},
{
"case_id": 4079,
"assignedto_id": 5,
"comment": "No result, just a comment"
}
]
}[
{
"id": 8658,
"test_id": 18577,
"status_id": 1,
"created_on": 1684937736,
"assignedto_id": null,
"comment": "This test passed",
"version": null,
"elapsed": "5m",
"defects": null,
"created_by": 18,
"custom_step_results": null,
"custom_testrail_bdd_scenario_results": null,
"custom_failure_severity": null,
"attachment_ids": []
},
{
"id": 8659,
"test_id": 18578,
"status_id": 5,
"created_on": 1684937736,
"assignedto_id": 5,
"comment": "This test failed",
"version": null,
"elapsed": "3m",
"defects": "TR-7",
"created_by": 18,
"custom_step_results": null,
"custom_testrail_bdd_scenario_results": null,
"custom_failure_severity": 3,
"attachment_ids": []
},
{
"id": 8660,
"test_id": 18579,
"status_id": null,
"created_on": 1684937736,
"assignedto_id": 5,
"comment": "No result, just a comment",
"version": null,
"elapsed": null,
"defects": null,
"created_by": 18,
"custom_step_results": null,
"custom_testrail_bdd_scenario_results": null,
"custom_failure_severity": null,
"attachment_ids": []
}
]Adding results with separated test steps
Separated test steps are a powerful mechanism supported by TestRail to have a more granular control of your test results. Besides setting a general test result, you can also set the a result for each step, and in case the step failed, you can submit the actual behavior. You can submit step results using a body content similar to the one below.
POST index.php?/api/v2/add_results_for_cases/{run_id}
curl -X POST \
-u "user@example.com:password" \
-F "@path/to/data.json" \
"https://example.testrail.io/index.php?/api/v2/add_results_for_cases/228"
{
"results": [
{
"case_id": 4078,
"status_id": 5,
"comment": "This test failed. See steps results below.",
"elapsed": "3m",
"environment": "qa03",
"custom_step_results": [
{
"content": "Step 1",
"expected": "Expected Result 1",
"status_id": 1
},
{
"content": "Step 2",
"expected": "Expected Result 2",
"actual": "Actual Result 2",
"status_id": 5
}
]
}
]
}[
{
"id": 8661,
"test_id": 18578,
"status_id": 5,
"created_on": 1684938025,
"assignedto_id": null,
"comment": "This test failed. See steps results below.",
"version": null,
"elapsed": "3m",
"defects": null,
"created_by": 18,
"custom_step_results": [
{
"content": "Step 1",
"expected": "Expected Result 1",
"actual": "",
"additional_info": "",
"refs": "",
"status_id": "1"
},
{
"content": "Step 2",
"expected": "Expected Result 2",
"actual": "Actual Result 2",
"additional_info": "",
"refs": "",
"status_id": "5"
}
],
"custom_testrail_bdd_scenario_results": null,
"custom_failure_severity": null,
"attachment_ids": []
}
]Importing attachments
Attachments are very useful to provide additional context for your test results, specially when tests fail. You can add any type of attachment, such as screenshots, videos, execution logs, among others.
Note that the maximum allowed upload size is set to 256MB.
POST index.php?/api/v2/add_attachment_to_result/{result_id}Content-Type: multipart/form-data
curl -X POST \
-u "user@example.com:password" \
-H "Content-Type: multipart/form-data" \
-F "attachment=@C:\\image.jpg" \
"https://example.testrail.io/index.php?/api/v2/add_attachment_to_result/8661"
{ "attachment_id": "758c9b26-6038-4147-9a4d-74cd7af1499a" }Closing a test run
In case you don’t want your imported test runs and results to be modified by anyone, you can also close them by using the following endpoint.
POST index.php?/api/v2/close_run/{run_id}
curl -X POST \
-u "user@example.com:password" \
-H "Content-Type: application/json" \
"https://example.testrail.io/index.php?/api/v2/close_run/228"
{
"id": 228,
"suite_id": 53,
"name": "New test run",
"description": "Test run description",
"milestone_id": null,
"assignedto_id": 5,
"include_all": false,
"is_completed": true,
"completed_on": 1684939401,
"config": null,
"config_ids": [],
"passed_count": 0,
"blocked_count": 0,
"untested_count": 1,
"retest_count": 0,
"failed_count": 2,
"custom_status1_count": 0,
"custom_status2_count": 0,
"custom_status3_count": 0,
"custom_status4_count": 0,
"custom_status5_count": 0,
"custom_status6_count": 0,
"custom_status7_count": 0,
"project_id": 27,
"plan_id": null,
"created_on": 1684936873,
"updated_on": 1684936873,
"refs": null,
"dataset_id": null,
"created_by": 18,
"url": "https://marketing80testing.testrail-staging.com/index.php?/runs/view/228"
}