With Webhooks, you can provide other applications with real-time information based on events that happen in TestRail.
Unlike typical API requests, where you would need to poll for data frequently in order to get new information as it is updated, webhooks let you send notifications and payloads to external systems or applications when certain events occur in TestRail like when your team adds new test cases, creates new test runs, or uploads new test results. This allows you to integrate TestRail with any application with an API, speed up key processes in your test cycle, and increase visibility into the progress of your testing activities for your entire team.
How to add a webhook
There are two different ways to add a webhook: globally (which will be triggered for any projects in your TestRail instance), or for specific projects.
To set up a webhook across all of your projects, go to Administration and click on Integration.
Choose the Webhooks tab. This is where you can add webhooks or edit existing ones.
The status column indicates if the webhook is currently active and working. If the webhook is receiving an error, the status will be flagged red. This overview also shows you the name of the webhook, the projects they are associated with, and the events that trigger the webhook.
Click Add Webhook to set up a new one. You will see the following window, which will allow you to add a name, specify payload URL, method, content type, and key information like headers and payload.
- Give your webhook a descriptive name.
- Add a Payload URL: This URL is where the webhook content will be sent to when it is triggered.
- Method: Choose between POST, GET, PUT, PATCH, or DELETE.
- Select a Content Type. Available options are:
- application/json
- application/x-www-form-urlencoded
- application/xml
- text/xml
- text/plain
- Add any additional Headers you may need for your webhook. All webhooks in TestRail have some default Headers that are in gray, but you can also add custom headers that will appear in black.
- Add a Payload to your webhook. The Payload field is designed to be flexible so that you can construct or send almost any kind of payload you wish using the console. (See Payloads below for more information.)
- You can also set a Secret to your webhook for authentication purposes, if needed. This will be included in the webhook headers.
- Under Events, select which events you would like to trigger webhooks by checking the boxes to the left of each option
- You can choose multiple events to trigger the same webhook
- Under Projects, check the boxes for each project where you would like your selected events to trigger this webhook.
- Finally, you can activate or deactivate the webhook by checking (or unchecking) the Active box.
- Click Save Settings when you’re ready.
Once you have configured a webhook, you can test the webhook settings using the Test button in the top right-hand side of the Add Webhook dialogue. Learn more about testing your webhooks configuration here.
Adding webhooks to specific projects
You can also add webhooks to specific projects via the Edit Project page. Go to Administration, click Projects from the menu in the right sidebar, and then click the pencil icon to go to the Edit Project page. Or, if you are already on the Project Overview page for a specific project, click Edit in the top right corner. Once you are on the Edit Projects page, click the Webhooks tab to add or edit a webhook specific to that project.
Testing and auditing webhooks
When adding a new webhook, it’s a good idea to test it before saving it so you can first review the request, payload, and response. You can test webhooks while initially configuring them in the Add Webhooks page, or return to an existing webhook to test it later. You can also review historical requests triggered by webhooks that you have set up in your Webhooks settings.
To test a webhook you have already created or review historical activity, navigate back to Integration under the Administration tab, and return to the Webhooks tab. Locate the Testing section at the right hand of the page.
Here, you can see a log of the recent deliveries and the dates. Click on a recent delivery to review the information that was included in the headers and the payload of your webhook based on the specific event that triggered that delivery under the Request tab.
On the Response tab, you can see the status code for the response that TestRail received from your external application which received the webhook, as well as the headers and payload of the response.
Webhook events
When you set up a webhook in TestRail, you can choose specific events that will trigger it. Here is a list of the events that you can select to trigger webhooks at this time.
Event | Description |
---|---|
Plan created | If you check this event, the webhook will be triggered every time a new Test Plan is created in the selected project/s. |
Plan updated | If you check this event, the webhook will be triggered every time there is an update to the Test Plan in the selected project/s |
Plan Entry created | If you check this event, the webhook is triggered every time a Test Plan entry is created in the selected project/s. |
Plan Entry updated | If you check this event, the webhook will be triggered every time a Test Plan entry is updated in the selected project/s. |
Run created | If you check this event, the webhook will be triggered every time a Test Run is created in the selected project/s |
Run updated | If you check this event, the webhook will be triggered every time a Test Run is updated in the selected project/s |
Case created | If you check this event, the webhook will be triggered every time a Test Case is created in the selected project/s |
Case updated | If you check this event, the webhook will be triggered every time a Test Case is updated in the selected project/s |
Test result created | If you check this event, the webhook will be triggered every time a Test Result is created in the selected project/s |
Report created | If you check this event, the webhook will be triggered every time a Report is created in the selected project/s. |
All | If you check this box, the webhook will be triggered every time any of the previous scenarios happens. |
You can add a number of the variables to your payload to add more context and data to your webhook. See the full list below.
Payloads
The payload field available in the webhook console lets you construct and send almost any kind of payload you wish. This is an example of JSON payload structure to know the kind of things that can be done for Slack.
{
"text": "%event_creator% created a new test case:",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*%event_creator% created a new test case:*"
}
},
{
"type": "divider"
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*<%url%|%name%>* \n Priority: %case_priority% \n Type: %case_type%"
}
}
]
}
When the webhook is executed, the result is a message similar to this one in slack.
Payload variables
Different variables are available to supplement payloads. The variables available for use in webhooks payloads at this time are:
Variable | Type | Description |
---|---|---|
%assigned_to% | text | Used for the name of the user to which the entity is assigned. |
%case_priority% | text | Used for the test case priority field status when triggering a webhook payload relating to a test case event (e.g. “High”, “Medium” “Low”). |
%case_type% | text | Used for the type of test case for which the event payload was triggered, if specified (e.g. “Functional,” “Smoke,” “Regression”, etc). |
%completed_on% | timestamp | Used for the milestone/plan/run completion (marked as ‘closed’) timestamp. |
%config% | text | Used for the entity config name. (Test plan entries only). |
%custom_x% | object/array | Used to return all of the custom test case fields, when a test case event is triggered, with the field starting with “custom_” (i.e. replace “x” with the custom field’s names). (E.g. If you made a custom field called ‘Version number’, the variable should be %custom_version_number%). |
%description% | text | Used for the description of the entity for which an event will trigger a webhook payload (e.g. milestone description or test run description). |
%due_on% | timestamp | Used for the date on which a milestone is expected to complete. (Milestones only). |
%entity_created% | timestamp | Used for the time at which the entity (for which the event was triggered) was created. |
%entity_creator% | text | Used for the username who created the entity, for which the event was triggered (e.g. the username who created the test case originally). |
%estimate% | integer | Used for the estimate field of a test case when triggering a webhook payload relating to a test case event. |
%event_created% | timestamp | Used to specify the time at which the webhook event occurred. |
%event_creator% | text | Used for the username that created and triggered the webhook event. |
%event_type% | text | Used for the type of event which triggered the webhook payload (e.g. “Plan created”, “Case updated”, “Report created”, etc). |
%id% | integer | Used for the ID of the entity for which an event will trigger a webhook payload (e.g. “117” in the case ID label “C117” or “42” in the run ID label “R42”). |
%is_deleted% | boolean | Used to indicate whether or not the case, run, plan, or milestone has been marked as deleted. |
%milestone_id% | integer | Used for the unique ID of the milestone with which the entity is related, if associated to any milestones. |
%more_info% | text | Used for the API endpoint which can be called to find out more information about the entity for which the event was triggered (e.g. “/api/v2/get_runs/3”). |
%name% | text | Used for the name (or the title) of the entity for which an event will trigger a webhook payload. |
%project_id% | integer | Used for the TestRail project to which the entity is related. |
%refs% | text | Used for any information stored in the “references” field of the webhook entity (e.g. “TR-71, TR-72”). |
%section_id% | integer | Used to identify the section associated with a test case. |
%stats% | object/array | Used for the complete set of test results metrics relating to a plan, run or milestone (passed_count, blocked_count, untested_count, retest_count, failed_count, custom_status_n_count). |
%suite_id% | integer | Used for the identification of the test suite related to the webhook entity subscription. |
%template_name% | text | Used for the test case template used when triggering a webhook payload relating to a test case event (e.g. “Test Case Steps,” “Exploratory Session”). |
%url% | text | Used for the instance URL via which the entity can be viewed in TestRail (e.g: “https://allboard.testrail.com/index.php?/milestones/view/61″). |
%secret% | text | Used for the token entered in the Secret field of the webhook, if it exists. |
You can add the variables to the payload field and they will be populated with the corresponding TestRail data when the webhook is triggered like in the image below.
On-premise installation (TestRail Server)
Installing the TestRail Server version of TestRail 7.5 involves some additional, optional, steps directly related to the new webhooks features. During the installation or upgrade process you’ll be presented with an additional step during which you can configure a RabbitMQ message queue server to handle your outbound webhook payloads.
Using a messaging queue server to handle outbound webhook payloads for TestRail Server instances is optional, and not required for installation or upgrade to TestRail Server 7.5. If you don’t want to add a message queue server during the installation or upgrade process, you can just simply skip to the next step of the installation process by clicking the Next button in the installation wizard.
If you decide you do want to use a message queue for this purpose, then during your TestRail Server installation or upgrade, you’ll need to add:
- The RabbitMQ server name
- The RabbitMQ port number (5672 by default)
- The RabbitMQ username and password
- And optionally the RabbitMQ certificate and private key
The message queue settings can also be found and added or updated in the Administration > Site Settings console.
Only RabbitMQ is supported as a message queue server at this time.