Overview
The Status Tops (Cases) report lets you see which test cases are used the most over time or which ones have high failure rates. By reviewing status tops, you can do a quick pulse check on which features are most problematic and where your test failures are occurring most often.
Creating a Status Tops (Cases) report
Prerequisites
Please make sure you have the following permissions enabled for your user role.
For more details, please check out the managing user roles and permissions guide
Steps to create a Status Tops (Cases) report
To generate the Status Tops (Cases) report in TestRail, you need to put a few configuration settings in place.
Navigate to the project for which you would like to create a test report. Then, navigate to the Report tab.
On the right side, you will see a pane titled Create a Report. Click on Status Tops report under the Cases section.
On Add Report page, you will find 3 main sections:
- Name and Description
- Report Options
- Access & Scheduling
Follow the below steps to configure each section
- Name and Description
Enter the report's name and description in their respective fields.
- Report Options
On the Add Report page, scroll down to Report Options. Here, you will see three tabs, STATUSES & RESULTS, SECTIONS & TEST RUNS, and TEST CASES.
STATUSES & RESULTS
The STATUSES & RESULTS tab allows you to filter which test statuses you want to see in your report. The default is All statuses. However, you can select the specific statuses you want to report. Select The following statuses only and then highlight your desired statuses. You can select multiple statuses to further focus your Status Tops report in your area of interest by holding Ctrl/Cmd on the keyboard to highlight your preferences.
Also, under this tab, you can also choose which test results to include. The template default is All test results of the tests, which will show you all of the statuses of test results run against your all-time test cases. You can change this to The latest test results per test only if you are looking to focus on the status of the latest test that was run against your test cases.
SECTIONS & TEST RUNS
-
You can do the following under the sections and test runs tab:
- Choose to include all sections of test cases or specific sections only. This allows you to show the results for specific sections of your test case repository that you would like to display in your report. For instance, if you would like to focus on particular features like Login and Messaging, you can select those two sections alone. Hold down the ctrl/cmd key and click to select multiple sections or sub-sections.
- Choose the test runs for which you would like to view test results. This can be done in two ways. The first option is to apply one or more filters to select the test runs that match the applied filter(s). You can filter the test runs using parameters like Assigned To, Completed On, Created By, Created On, Is Completed, Milestone, and Test Plan. This can be helpful if you want to compare results for tests across multiple test runs that fit certain criteria, e.g. all the test runs in a given test plan, or across a milestone that you are using to track testing for a specific release.
- Afterwards, click Add Test Runs; then, click Apply Filters.
The second option is to make individual selections of the test runs, one by one.
Select the maximum number of test runs to include. The maximum number can be five, 10, 25, 50, or 100.
TEST CASES
You can do the following under the TEST CASES tab:
- Choose the column(s) to include in the table to display information about test cases. The default columns include ID and Title. Depending on the information you want to display, you can add additional columns such as Status, Type (of the test case), Priority, etc.
- Afterwards, you can set the maximum number of test cases to include in the report. The maximum number can be 100, 250, 500, 1000, 2500, or 5000.
-
- Access & Scheduling
This section enables users to set access and scheduling options. For more details, please refer to the General configurations guide.
Finally, scroll down and click Add Report to generate the report.
Reading a Status Tops (Cases) report
It may take minutes to generate your report depending on the data included. However, once the report is fully generated, the next thing is to view and interpret it. To view the report, go to your Reports tab and click on the appropriate report under the list.
The first chart on the report page is a bar graph depicting the statuses you requested when configuring the report. The number of test results that have been recorded with each status in the test runs included in the report show along the x-axis. Your customized status names and colors will be reflected in the bar graph.
Under the bar graph, you will see a table listing the test runs you included in the report. In this table, the title of the run hyperlinks to more detail. This will take you to a new screen with further details (unless you previously checked the box Do not include links in your Report Options). To return the individual test case details to your report, use the back arrow in your browser window. The test run’s test plan affiliation is in the middle column and also hyperlinked to more detail. The percentage in the far right column reflects the number of passing tests in that test run.
Below the Test Runs table, you’ll also see a Statuses table listing relevant test cases that have been added to the test runs in your report, grouped by the status of the most recent test against that test case. You can learn more about the test case by clicking on the hyperlinked title text in the left-hand column (if links were included). You may also see a grouping with No test cases found with this status. This indicates that the test runs selected had no test cases meeting that status criterion.