1. TestRail Support Center
  2. User Guide
  3. Reporting and analysis
  4. Defects reports

Summary for References (Defects) report

Updated

Overview

The Summary for References (Defects) report shows defects recorded and linked to test runs in TestRail for test cases, categorized by reference. This report provides insight into the test status related to specific references and the quality of the corresponding system capabilities and provides traceability between references, test cases, test results, and defects identified. The report includes a graphical representation of defects by test run, as well as a detailed breakdown of defects by reference over a series of test runs.

Creating a Summary for References (Defects) report

Prerequisites

Please make sure you have the following permissions enabled for your user role.

Reports_Permission.png

For more details, please check out the managing user roles and permissions guide

Steps to create a Summary for References (Defects) report

To generate the Summary for References (Defects) report in TestRail, you need to put a few configuration settings in place.

Navigate to the project for which you would like to create a test report. Then, navigate to the Report tab.

On the right side, you will see a pane titled Create a Report. Click on Summary for References under the Defects section.

Defects_SummaryForCases_Select.png

On Add Report page, you will find 3 main sections: 

  • Name and Description
  • Report Options
  • Access & Scheduling

Follow the below steps to configure each section

  1. Name and Description

    Enter the report's name and description in their respective fields.

  2. Report Options

    On the Add Report page, scroll down to Report Options. Here, you will see three tabs, REFERENCES, SECTIONS & TEST RUNS, and TEST CASES.

    REFERENCES 

    Under the REFERENCES tab, you have two options. The first is to include all references, thereby generating a report for all the references you have linked to your tests.

    ReportOptions_ReferencesTab.png

    The second option is to add the reference IDs of the specific reference(s) you would like to include in your report. You should enter only one reference ID per line.

    ReportOptions_SpecificReferences.png

     

    SECTIONS & TEST RUNS

    You can do the following under the SECTIONS & TEST RUNS tab:

    CoverageForReferences_SectionsAndTestRuns.png

    1. Choose to include all sections of test cases or specific sections only. This allows you to show the results for specific sections of the test case that you would like to display in your report. Hold down the ctrl/cmd key and click to select multiple sections or sub-sections.

      CoverageForReferences_SelectMultipleSections.png
    2. Choose the test runs for which you would like to view test results. This can be done in two ways. The first option is to apply one or more filters to select the test runs that match the applied filter(s). You can filter the test runs using parameters like Assigned To, Completed On, Created By, Milestone, etc. This can be helpful if you want to compare results for tests across multiple test runs that fit certain criteria, e.g. all the test runs in a given test plan, or across a milestone that you are using to track testing for a specific release.

      CoverageForReferences_SelectTestRuns.png

       

      Afterwards, click Add Test Runs; then, click Apply Filters.

       ApplyFilter.png

      The second option is to make individual selections of the test runs, one by one. Click the radio button next to Only the following test runs, then click Add Test Runs and check the boxes for each test run you would like to include in your report.

      CoverageForReferences_AddSpecificRuns.png

    3. Select the maximum number of test runs to include. The maximum number can be five, 10, 25, 50, or 100. 

    TEST CASES 

    You can do the following under the TEST CASES tab:

    1. Choose the column(s) to include in the table to display information about test cases. The default columns include ID and Title. Depending on the information you want to display, you can add additional columns such as Status, Type (of the test case), Priority, etc.

      CovereageForReferences_AddColumn.png
    2. If you would like to show the comparison between the test results for each selected test run in your report, you need to check the box provided for the option as shown in the Figure below. The same applies if you want to show the latest/combined test result for the selected test runs. Afterwards, you can set the maximum number of test cases to include in the report. The maximum number can be 100, 250, 500, 1000, 2500, or 5000.

      TestCasesTab_AdditionalDetails.png

     

  3. Access & Scheduling

    This section enables users to set access and scheduling options. For more details, please refer to the General configurations guide.

Finally, scroll down and click Add Report to generate the report.

Reading a Summary for References (Defects) report

At this juncture, it is important to note that generating your report may take time, depending on the amount of data included in the report. However, once the report is fully generated, you can view it by going to the Reports tab and clicking on the appropriate report under the list.

The first chart on the report page is a bar graph that illustrates the number of defects found during applicable test runs with references. The Combined (Summary) column on the far right shows the total number of linked defects from all test runs included in the report. Note that test cases without a reference are excluded from the report. The totals include only defects linked to test cases with references. An example of this graph is shown below. 

Defects_SummaryForCases_Read_image1.png

Next, you will see a list of test runs that have defects and meet the criteria provided in the report options. This list includes high-level information about each test run including the corresponding test section and milestone. An example is shown below. The note at the bottom indicates the number of test runs that met the search criteria but did not have any defects.

Defects_SummaryForCases_Read_image2.png

Afterwards, you will see a list of the test cases with references shown along with additional information about applicable test runs. Test cases with no references will not appear on the report. If a case was tested in a given run, the status of that test run is indicated by color. For example, a red oval indicates the test failed. A blank space in the table indicates that the test case was not included in the run or did not have any linked defects.

If you selected the comparison report option, a column for each test run is included that shows the current status of the test executed against the case and a list of defects recorded for that test run.

If you selected the summary report option, a summary column on the far right shows the most recent test status across all test runs and links to defects from that test run. Defects are hyperlinked to the corresponding entries in an integrated defect tracking system, if applicable. 

Defects_SummaryForCases_Read_image3.png

 

 

Was this article helpful?
0 out of 0 found this helpful