JUnit to TestRail mapping

JUnit report example

The JUnit report format is widely adopted by test automation frameworks and CI tools to save and visualize test results. The TestRail CLI parses these reports and converts the elements into TestRail entities, which are imported via the API. Below you can find a standard JUnit report example.

<testsuites name="test suites root">
  <testsuite failures="0" errors="0" skipped="1" tests="1" time="3049" name="tests.LoginTests">
    <properties>
      <property name="setting1" value="True"/>
      <property name="setting2" value="value2"/>
    </properties>
    <testcase classname="tests.LoginTests" name="test_case_1" time="159">
      <skipped type="pytest.skip" message="Please skip">skipped by user</skipped>
    </testcase>
    <testcase classname="tests.LoginTests" name="test_case_2" time="650">
    </testcase>
    <testcase classname="tests.LoginTests" name="test_case_3" time="121">
      <failure type="pytest.failure" message="Fail due to...">failed due to...</failure>
    </testcase>
  </testsuite>
</testsuites>

JUnit tags

When parsed by the TestRail CLI, each JUnit report tag is converted to a TestRail entity, as per the table below.

XML JUnit tags TestRail mapping
<testsuites> Suite
<testsuite> Section
<testcase> Test case

JUnit testsuite

Elements with the <testsuite> tag are converted into TestRail sections and their properties are added to the test run description on TestRail. Below you can see an example of a <testsuite> with <property> child tags.

<testsuites name="test suites root">
  <testsuite failures="0" errors="0" skipped="1" tests="1" time="3049" name="tests.LoginTests">
    <properties>
      <property name="setting1" value="True"/>
      <property name="setting2" value="value2"/>
    </properties>
    (...)
  </testsuite>
</testsuites>
Attribute or tag TestRail mapping
name Section name
<property> Property names and values are concatenated and added to test run description

JUnit testcase

Elements with the <testcase> tag are converted into TestRail test cases and respective test results. 

Attribute TestRail mapping
name Test case name
time Test result elapsed time

The test result status is defined by the child tags of the <testcase> tag. If there is no tag, the test result will be Passed, while if there is a <failure> tag the test result will be Failed, and if there is a <skipped> tag the result will be Retest. Examples can be seen on the table below.

XML structure TestRail status
<testcase classname="tests.LoginTests" name="test_case_1" time="135">
</testcase>
 Passed
<testcase classname="tests.LoginTests" name="test_case_1" time="135">
  <failure type="errorType" message="Failure message">
    Error (stacktrace)
  </failure>
</testcase>
 Failed
<testcase classname="tests.LoginTests" name="test_case_1" time="135">
  <skipped type="skipType" message="Skip test">
    Test skipped by user
  </skipped>
</testcase>
 Retest

The data in the attributes and text of the <failure> and <skipped> tags is added to the test result comment, in a format such as below.

Type: errorType
Message: Failure message
Text: Error (stacktrace)

JUnit testcase properties

You can enrich your test cases and test results data by adding custom properties to your reports. The supported properties and their meaning are listed on the table below.

Name TestRail mapping
testrail_case_field Test case field
testrail_result_field Test result field
testrail_result_comment Append value to test result comment
testrail_result_step Add a step and its status to the result
testrail_attachment Upload attachment in path to test result (max 256MB)

While some of these properties only require a simple value, field properties require the name of the field and the respective value. For these properties, you should use the format field_name:field_value. For more information on how to use these properties, please see the TestRail CLI usage examples page.

<testsuites name="test suites root">
  <testsuite failures="0" errors="0" skipped="1" tests="1" time="3049" name="tests.LoginTests">
    <testcase classname="tests.LoginTests" name="test_case_1" time="650">
      <properties>
        <property name="testrail_case_field" value="custom_preconds:My preconditions"/>
        <property name="testrail_case_field" value="type_id:3"/>
        <property name="testrail_result_field" value="version:1.1"/>
        <property name="testrail_result_field" value="custom_field:custom_value"/>
        <property name="testrail_result_step" value="passed: Insert login credentials"/>
        <property name="testrail_result_step" value="failed: Click submit"/>
<property name="testrail_result_step" value="untested: User should be logged in"/>
<property name="testrail_result_comment" value="Finding 1"/>
<property name="testrail_result_comment" value="Finding 2"/> <property name="testrail_attachment" value="path_to/logs.log"/> <property name="testrail_attachment" value="path_to/screenshot.jpg"/> </properties> </testcase> </testsuite> </testsuites>

 

Was this article helpful?
6 out of 14 found this helpful