The API use cases section is meant for TestRail users and third-party app developers with the goal of creating an integration with TestRail. It describes how to use the TestRail API for multiple use cases in order to integrate your tools with TestRail, along with useful examples and technical implementation samples.
Test cases and test results are often the most relevant entities when it comes to test management. Using the TestRail API, among a lot of other things, you should be able to:
Before going any further into implementation details, we recommend you make yourself familiar with TestRail entities, customizability and limitations, which are described in the sections below.
TestRail entities
This section provides an overview of the data entities available in TestRail, as well as their relationships and particularities. These are the elements you will want to look into when creating your integrations.
| Entity | Description |
|---|---|
| Project |
The project is the highest level organizational unit in TestRail. Use projects to separate your efforts by product, project team, etc. |
| Suite |
Suites (or test suites) can be used in multi-suite type projects to provide an extra layer of separation between test entities, as if you had multiple projects in one. Important notes:
|
| Sections | Sections are used to organize cases and can have multiple levels of hierarchy for flexibility. Any case you create will be associated with a section. |
| Cases | Cases (or test cases) are the entities used to specify the procedures of your tests. They contain various information about what you aim to test, such as preconditions, steps and all types of other test attributes. Case fields and templates can be fully customized. |
| Runs | Runs (or test runs) are entities used to aggregate test cases for execution. Runs aggregate tests and results. |
| Plans | Plans aggregate runs in plan entries and have the ability to further associate them with configuration groups. |
| Tests | Tests are the materialization of cases for execution inside runs. A case can relate to multiple tests, executed in different test runs. Tests aggregate results. |
| Results | Results determine the status (i.e.: passed, failed, etc) of a test, as well as other information, such as comments and defects. Result fields can be fully customized. |
| Test steps | Test steps are generated when cases are created with a separated steps template, allowing you to specify steps and results in a more granular manner. There is a particular format to manage these in case and result entities, which enables you to submit step-by-step results. |
Understanding TestRail structure
Projects
TestRail has multiple project types, as per the image below. When it comes to API usage, what this means is that for projects which are not single repository, you have to be aware of the separation between suite entities.
Separation between suite entities
Entities in one suite can’t be associated to an entity in a different suite. For API bulk operations, all entities must belong to the same suite. For example, you can't select test cases from two different test suites for one test run, as the test run itself is bound to one suite.
Test design entities example
When it comes to test design, you can organize your test cases in sections and subsections. Sections and subsections are the same entity, the different is that a subsection contains a parent section.
Test cases can make use of custom fields and different templates, according to your configurations, which you have to be aware of and contemplate in your integrations.
Test execution entities example
To submit test results, tests must be added to test runs, which are the basic container for tests. For organization purposes, you can optionally leverage test plans and milestones to aggregate test runs with similar purposes.
Relationship between test design and execution
Test cases work as a template for tests. Tests live inside test runs and reflect the execution of cases, aggregating one or more results.
Customizing TestRail
TestRail aims to be very flexible to provide support for all kinds of testing workflows. To accommodate custom data required for these workflows, TestRail supports the following field customizations:
- Case fields - Add/remove or modify any data field in your cases
- Case templates - The template you use to write your test cases (i.e.: separated steps)
- Result fields - Add/remove or modify any data field in your results
- Result statuses - Add/remove or modify the default statuses and their configurations
Note that customizations can be applied to all projects or on a project-by-project basis.