Overview
The Coverage for References (Cases) report offers you the ability to get a quick glance into test coverage for references in a coverage matrix. References are used to link your development team requirements in test cases within TestRail.
To run the Coverage for References (Cases) report, you will first need to have references identified in test cases created by your users. With this report, you can quickly do a pulse check on test coverage across all references or a specific set of references. This helps you verify that test cases have been written for all the specified development team requirements.
Creating a Coverage for References (Cases) report
Prerequisites
Please make sure you have the following permissions enabled for your user role.
For more details, please check out the managing user roles and permissions guide.
Steps to create a Coverage for References (Cases) report
To generate the Coverage for References (Cases) in TestRail, you need to put a few configuration settings in place.
Navigate to the project for which you would like to create a test report. Then, navigate to the Report tab.
On the right side, you will see a pane titled Create a Report. Click on Coverage for References under the Cases section.
On Add Report page, you will find 3 main sections:
- Name and Description
- Report Options
- Access & Scheduling
Follow the below steps to configure each section
- Name and Description
Enter the report's name and description in their respective fields.
- Report Options
On the Add Report page, scroll down to Report Options. Here, you will see three tabs, REFERENCES, SECTIONS, and TEST CASES.
REFERENCES
Under the REFERENCES tab, you have two options. The first is to include all references, thereby generating a report for all the references you have linked to your tests.
The second option is to add the reference IDs of the specific reference(s) you would like to include in your report. You should enter only one reference ID per line.
SECTIONS
You can do the following under the SECTIONS tab:
Choose to include all sections of test cases or specific sections only. This allows you to show the results for specific sections of the test case that you would like to display in your report. Hold down the ctrl/cmd key and click to select multiple sections or sub-sections.
TEST CASES
You can do the following under the TEST CASES tab:
-
Set additional filters and select what data from your test cases you want to display in the report.
The report by default will display both Test cases with references and Test cases without references.
By removing the check from Test cases with references, you could run a report identifying potential gaps in the testing coverage.
- Choose the column(s) to include in the table to display information about test cases. The default columns include ID and Title. Depending on the information you want to display, you can add additional columns such as Status, Type (of the test case), Priority, etc. Afterwards, you can set the maximum number of test cases to include in the report. The maximum number can be 100, 250, 500, 1000, 2500, or 5000.
-
- Access & Scheduling
This section enables users to set access and scheduling options. For more details, please refer to the General configurations guide.
Finally, scroll down and click Add Report to generate the report.
Reading a Coverage for References (Cases) report
At this juncture, it is important to note that generating your report may take time, depending on the amount of data included in the report. However, once the report is fully generated, you can view it by going to the Reports tab and clicking on the appropriate report under the list.
The first chart on the report page is a pie chart and bar graph depicting your requested references information. The pie chart shows the percentage of reference coverage for the test cases. If you selected All references when configuring the report, you will see a completely blue circle for 100% coverage. Or if each of the reference IDs you pasted in the specific references section is covered in a test case, you will also see the 100% coverage blue circle. If nothing is covered, you will see an entirely black circle for 0% coverage.
The bar graph summarizes the references, number of linked test cases, and/or test cases that don’t have any listed references. At the top, the green bar represents the number of references that you have added to the test cases. The middle, orange bar reflects the number of cases with references. The final, blue bar shows the number of cases without references.
Under the bar graph, you will see a table listing the References, Test cases with references (if there are any), and Test cases without references.
The References table will include the information you requested in setting up your reports such as the ID number, test case title, and priority. You can at this point, link on the individual hyperlinked reference to get more details. For instance, if the reference is in Jira, you will be taken to that software to learn more. If you are not logged into Jira at the time, you will first be asked to log in.
Next, in the table, you will find a listing of the test cases with and/or without references. Remember, you have the option of asking for both in the report or just one of either. The test cases will be grouped by reference if a reference exists. If test cases appear without references, you may want to review them and add the corresponding reference ID by editing the test case to improve traceability between your requirements and testing efforts recorded in TestRail.
The test cases will be listed with hyperlinks to the more detailed test case record within TestRail. If there is reporting on subsections within the test case, those subsections will be indented slightly to indicate the section hierarchy.