Code-first workflow

The code-first workflow is ideal for technical teams who manage test design directly in their codebase. It allows you to sync automated test results with TestRail without manually creating or managing test cases in the TestRail UI. 

This approach keeps your test management lightweight and integrated into your CI/CD pipeline while still giving your team visibility and reporting capabilities via TestRail.

Why use Code-First Workflow?

• Avoids the overhead of duplicating test cases in TestRail
• Keeps test design and logic in your automation code
• Automatically syncs and creates test cases based on your test execution
• Ideal for engineering teams who "code their tests first" before managing them in a tool

Code-First vs. Specification-First differences

Uploading Test Results using the CLI

Prerequisites

Before you start uploading test results:

1. Install TestRail CLI: Follow the TestRail CLI installation guide.
2. Add a custom field to TestRail:
   • Go to Admin > Customizations > Add Field
   • Create a field named (System name) automation_id
   • Field type: String
   • Applies to: Test Cases
   • This field is used to map code-based tests to TestRail test cases.

Upload the Test Results

1. Prepare your JUnit XML Report

Here’s a sample JUnit-style test result file:

<testsuites name="test suites root">
  <testsuite failures="0" errors="0" skipped="1" tests="1" time="0.05" name="tests.LoginTests">
    <testcase classname="tests.LoginTests" name="test_case_1" time="159">
      <skipped message="Please skip">skipped by user</skipped>
    </testcase>
    <testcase classname="tests.LoginTests" name="test_case_2" time="650"/>
    <testcase classname="tests.LoginTests" name="test_case_3" time="159">
      <failure message="Fail due to...">failed due to…</failure>
    </testcase>
  </testsuite>
</testsuites>

2. Upload with the CLI

This command uploads the test results and creates test cases automatically if needed:

# Uploads results.xml to TestRail using code-first mapping
trcli -y \
  -h https://<INSTANCE-NAME>.testrail.io \
  --project "TRCLI Test" \
  --username <YOUR_EMAIL> \
  --password <API_KEY_OR_PASSWORD> \
  parse_junit \
  --title "Automated Tests Run" \
  -f results.xml

Parameters explained:

• -y: Skips confirmation prompts, ideal for CI usage
• --project: Your TestRail project name
• --title: Name for the test run
• -f: Path to your JUnit XML results file

How the mapping works

Each test case is mapped using the automation_id field, which is automatically generated from the test class and name in the format:

<classname>.<testname>

For example,

<testcase classname="tests.LoginTests" name="test_case_1"/>

Results in the automation_id:

tests.LoginTests.test_case_1

TestRail will match this against existing cases using the automation_id. If it can’t find one, it will create a new test case for you.

Understanding Test Case Mapping via automation_id

One of the core features of the code-first workflow is automatic test case mapping using the automation_id field in TestRail. This allows your automation code and TestRail test cases to stay in sync—without needing to manually add TestRail case IDs to your test code.

Each time you upload results with the CLI, it tries to match test cases based on this automation_id. If it doesn’t find a match, it creates a new test case automatically.

Take the sample JUnit XML snippet below. The CLI combines the classname and name attributes to build a unique automation ID for each test:

<testcase classname="tests.LoginTests" name="test_login_with_invalid_password" time="159"/>
<testcase classname="tests.LoginTests" name="test_login_with_valid_credentials" time="221"/>

This results in the following automation_id values:

tests.LoginTests.test_login_with_invalid_password
tests.LoginTests.test_login_with_valid_credentials

When you run the CLI, it matches these strings against the automation_id field in your TestRail test cases:

$ trcli -y \
  -h https://INSTANCE-NAME.testrail.io \
  --project "Login Tests" \
  --username user@domain.com \
  --password passwordORapikey \
  parse_junit \
  --title "Login Feature Tests" \
  -f login_tests_results.xml

CLI output:

Parsing JUnit report.
Processed 2 test cases in 1 section.
Found 1 matching TestRail case.
Adding 1 new test case to the suite.
Creating test run. Run created: https://INSTANCE-NAME.testrail.io/index.php?/runs/view/456
Adding results: 2/2, Done.

In this example:

• One test case was already mapped via automation_id
• One test case didn’t exist and was created automatically
• Both results were uploaded in a single run

Mapping Lifecycle Example

Let’s say you later refactor a test:

- <testcase classname="tests.LoginTests" name="test_login_with_valid_credentials"/>
+ <testcase classname="tests.AuthTests" name="test_login_successful"/>

The new automation ID becomes:

tests.AuthTests.test_login_successful

If you upload this result without updating the automation_id in TestRail, the CLI will create a new test case, not update the existing one.

Mapping from Real JUnit Report

Let’s take a closer look at this real-world JUnit XML snippet and how the TestRail CLI interprets it:

<testsuites name="test suites root">
  <testsuite failures="0" errors="0" skipped="1" tests="1" time="0.05" name="tests.LoginTests">
    <testcase classname="tests.LoginTests" name="test_case_1" time="159">
      <skipped type="pytest.skip" message="Please skip">
        skipped by user
      </skipped>
    </testcase>
    <testcase classname="tests.LoginTests" name="test_case_2" time="650"/>
    <testcase classname="tests.LoginTests" name="test_case_3" time="159">
      <failure type="pytest.failure" message="Fail due to...">
        failed due to…
      </failure>
    </testcase>
  </testsuite>
</testsuites>

The CLI will generate the following automation IDs:

You can then run this command to push results to TestRail:

trcli -y \
  -h https://INSTANCE-NAME.testrail.io \
  --project "TRCLI Test" \
  --username user@domain.com \
  --password passwordORapikey \
  parse_junit \
  --title "Automated Tests Run" \
  -f results.xml

Result Summary from CLI

Parsing JUnit report.
Processed 3 test cases in 1 sections.
Found 3 test cases not matching any TestRail case.
Adding missing sections to the suite.
Adding missing test cases to the suite.
Adding test cases: 3/3, Done.
Creating test run. Run created: https://INSTANCE-NAME.testrail.io/index.php?/runs/view/123
Adding results: 3/3, Done.
Submitted 3 test results in 8.9 secs.

After execution:

• New test cases are created in your TestRail project

• Each one will have a unique automation_id matching the pattern
• Results, including skipped and failed states, are reflected in TestRail

You can now view them in Test Runs & Results or Test Cases, with traceability back to your automation suite.

Final Notes

• This workflow is optimal for teams who manage their QA process through code and need lightweight TestRail integration.
• Keep your automation_id values consistent and up-to-date for best results.
  • If you would like to upload automation results for test cases that already exist in TestRail, be sure to update the automation_id for those test cases before uploading your automation results
  • If you change the test name or location in your automation suite later, that will create a new test case in TestRail, unless you also update the automation_id field for the test case in TestRail
• You can review uploaded runs in TestRail > Test Runs & Results > Automated Tests Run.

Where to get help

• Official TRCLI documentation
• GitHub: testrail-cli
• TestRail Full System Documentation