TestRail helps teams streamline their testing processes, ensuring higher quality releases, and improving collaboration across all stakeholders.
One of TestRail's most powerful tools for enhancing test automation is TRCLI, the TestRail Command Line Interface. TRCLI boosts automation efficiency and helps teams manage their testing processes in a smarter, more streamlined way.
What is the TestRail CLI?
The TestRail CLI (TRCLI) is a command-line interface for TestRail that automates common operations, such as adding test results, creating test cases, or generating reports—all without needing to log into the web interface. This opens the door to powerful integration possibilities with test automation and CI/CD frameworks which will result in an enhanced efficiency.
For teams already working with automated testing frameworks, TRCLI can be a game changer. By integrating with the existing test automation scripts, TRCLI allows to push test results and interact with TestRail programmatically, giving teams the freedom to focus on other tasks that might require human capabilities and abilities.
Test automation efficiency enhanced
Seamless integration with automation frameworks
Most automation frameworks — like Selenium, Cypress, JUnit, TestNG, or Pytest — generate test results in various formats (JSON, XML, etc.).
TRCLI bridges the gap by providing an easy way to push those results directly into TestRail with a simple command. After the test suite finishes running, teams can use TRCLI to push results directly into TestRail with a simple command, eliminating the need for manual data entry or a separate reporting tool, ensuring that all the test results are centralized in TestRail for easy tracking.
With this integration, teams can view the status of their automated tests (which can be passed, failed, skipped, or blocked) directly in TestRail, creating a seamless workflow between automated tests and test management.
Automated test case creation and maintenance
As automation evolves, so does the suite of test cases. With TRCLI, it’s possible to programmatically create or update new test cases and test runs without manually going into TestRail’s web interface.
In test automation, the test management approach can follow different methods. TRCLI supports both 'code-first' and 'specification-first' approaches, making it flexible for different workflows. Whether creating test cases directly in code or documenting them in TestRail first, TRCLI ensures that test cases remain up-to-date with minimal manual effort, even in fast-paced development cycles.
The TestRail CLI has mechanisms to support using either of these approaches, independently of the chosen approach: to write tests in code first, without adding test cases to TestRail, or if automating tests based on test cases already documented in the platform.
This capability ensures that test cases stay up-to-date without requiring a significant amount of manual work, especially in fast-paced development cycles where test cases often need revisions.
Real-time test results reporting
One of the major pain points for testers is having to manually update and track the status of tests, although by using TRCLI it’s possible to automate this step, feeding the latest test results directly into TestRail in real time.
By running automated tests on a Continuous Integration (CI) pipeline - like Jenkins - TRCLI allows integration with CI/CD pipeline so after every test execution, results are immediately reported back to TestRail. In the documentation, it’s possible to access several examples for framework integrations, so you’ll have guidance during the integration.
A simple command can trigger a post-build action, so that the TestRail project will be automatically updated after every test cycle, ensuring real-time visibility of test status without the need for manual intervention.
Test run management
Managing test runs for large test suites can be time-consuming. TRCLI simplifies this by enabling teams to create and manage test automation runs directly from the command line. No more navigating through the web interface to set up a new test run; just use TRCLI to handle this task.
This functionality of TRCLI is especially useful for teams running multiple test runs at once or dealing with complex test structures. The command-line approach provides a faster, more flexible way to organize and manage test executions of industry standard automation frameworks.
Manual testers: get started with TRCLI
Although the TRCLI is primarily geared towards developers and automation engineers, manual testers can also gain significant benefits with some basic guidance, helping them integrate automation results and combine manual and automated testing efforts.
By learning foundational command-line concepts and exploring TestRail’s resources, manual testers can leverage TRCLI to enhance their workflows and boost productivity by integrating manual and automated testing strategies.
TestRail Academy
Exploring TestRail Academy and enrolling in the Test Automation and Integrations course can offer you valuable insights into how TestRail’s Command Line Interface (TRCLI) can enhance your testing workflows.
Taking the course will not only make your day-to-day tasks more efficient but will also future-proof your skills, giving you the foundation to leverage your skills and knowledge about the TestRail framework. TestRail Academy is free and is a great opportunity to expand your expertise and enhance your contributions to the testing process.
Basic command-line concepts
Then, it’s important to get comfortable with the basic command-line interface (CLI). TRCLI is a command-line tool, which means you'll interact with it via text-based commands, rather than a graphical user interface (GUI).
If you're new to the command line, it’s helpful to check out some basic tutorials on navigating directories, running commands, and working with terminal outputs. Additionally, the TestRail CLI automation samples repository offers a variety of usage examples for several industry-standard frameworks.
Understanding TRCLI setup
Once you're familiar with the command line, the next step is installing TRCLI. This is a straightforward process, and you can find step-by-step instructions on the official TestRail documentation. As part of this setup, you'll need to install TRCLI on your local machine or CI/CD server. You can find installation instructions on TestRail’s documentation site.
TRCLI documentation
The official documentation is a valuable resource for mastering TRCLI and unlocking its full potential. You’ll find comprehensive details on available commands and parameters here.
Explore advanced features such as:
- Test case creation and management
- Test run management
- Integration with third-party tools
To get started with TRCLI, try automating some small, repetitive tasks that you currently do manually in TestRail, such as adding test results or creating new test cases.
These small wins will help you get comfortable with the tool before diving into larger, more complex automation projects.
Working with automation engineers
If you’re working closely with automation engineers and/or a development team, you can collaborate with them to understand how TRCLI interacts and integrates with automation frameworks and CI/CD tools.
This collaboration will help you gain practical insights into how to integrate TRCLI into your testing workflows and increase your exposure to the broader automation practices used by your team.
As a manual tester, getting started with TRCLI can improve your testing workflows and introduce you to integrating automation with manual testing in TestRail. By adopting it, you can increase efficiency, automate repetitive tasks, and collaborate with automation engineers on more advanced testing practices—all while ensuring high-quality software delivery.
How can TRCLI support you
Whether automating the reporting of test results, managing test cases, or integrating TestRail with CI/CD pipelines, TRCLI offers a wide range of capabilities to efficiently test workflows.
By adopting TRCLI users can save time, lower human error, and ensure that the test management process is as efficient and effective as possible.
Whether you are an experienced tester or a developer who wants to learn more about automation, there are some things to consider if you want to maximize your TRCLI usage.
Leverage TRCLI in CI/CD pipelines
Integrating TRCLI into your CI/CD pipeline is a reliable way to boost automation efficiency. For every test suite execution, it’s possible to push the results directly to TestRail and keep an eye on the metrics and achievements. Whether using CI/CD tools - like Jenkins, GitLab CI, or CircleCI - it’s possible to easily integrate TRCLI commands into your pipeline scripts.
Keep scripts maintainable
When writing scripts that invoke the TRCLI, it’s important to maintain modular scripts that can be reused across multiple projects and parameterized to allow flexibility for credentials, changes in project identifiers, etc.
The below example shows parameterization of the TestRail CLI command in a yaml file within an Azure pipeline:
pip install trcli
trcli -y \
-h $(host) \
--project "$(project)" \
-u $(username) \
-p "$(user_password)" \
parse_junit \
-f "reports/junit-results.xml" \
--title "Cypress Automated Tests Executed From AzureDevOps Pipelines" \
--run-description "Pipeline Run: $(System.TeamFoundationCollectionUri)$(System.TeamProject)/_build/results?buildId=$(Build.BuildId)"
displayName: 'Upload Cypress Test Results Using TestRail CLI'
This allows the team to scale automation efforts more efficiently without having to rewrite common operations.