If you’re making a choice between whether you want to run your TestRail application in our hosted environment, or on-premise (Server), there’s a few considerations you should take into account. This document aims to address those considerations by giving you an overview of the TestRail Server installation footprint, the functional differences between TestRail-Server and TestRail-Cloud, and some information about our future plans for TestRail Server in terms of supported components and likely deprecations.
TestRail Cloud & Server Functional Differences
Sometimes due to various operational factors, there may be slight differences in functionality between our Cloud and Server versions of TestRail. The table below identifies the differences at feature level, up to our current Server version (8.0).
TestRail Cloud customers always get our newest features and functionality first, except on rare occasions where the functionality is specifically targeted at customers hosting TestRail on their own environments, such as PHP 8.1 compatibility.
| Feature | TestRail Version | ||||
| Server 7.0 | Server 7.4 | Server 7.5 | Server 8.0+ | Hosted (Cloud) | |
| Shared Test Steps | ✅ | ✅ | ✅ | ✅ | ✅ |
| Media Library | ❌ | ✅ | ✅ | ✅ | ✅ |
| Interactive Charts | ❌ | ✅ | ✅ | ✅ | ✅ |
| Test Case versioning & Review (Enterprise) | ❌ | ✅ | ✅ | ✅ | ✅ |
| Webhooks | ❌ | ❌ | ✅ | ✅ | ✅ |
| Single Sign on & Multi-Factor Authentication (Enterprise) | ❌ | ❌ | ✅ | ✅ | ✅ |
| PHP 8.1 compatibility | ❌ | ❌ | ❌ | ✅ | ✅ |
| Customizable Dashboard Charts | ❌ | ❌ | ❌ | ✅ | ✅ |
| Result Exclusions | ❌ | ❌ | ❌ | ✅ | ✅ |
| Improved Reporting | ❌ | ❌ | ❌ | ✅ | ✅ |
- TestRail Cloud has an API rate limit for performance. On TestRail Professional, the API rate limit is 180 requests per minute. On TestRail Enterprise it is 300 requests per minute. If you use heavy automation on TestRail Server this would be something to consider
- Custom reports & custom defect plugins are only possible on TestRail Server, as access to the file system is required.
- Authentication via AD/LDAP connections is currently only possible on TestRail Server. While SSO is possible on TestRail Cloud Enterprise via SAML 2.0, AD & LDAP connections for authentication are not yet supported.
TestRail Server Footprint
When deciding whether to choose Server or Cloud, you should consider the infrastructure and hosting requirements. The installation footprint for TestRail looks like the block diagram below, with some differences depending on whether you choose a Windows or Linux based environment. Specifically:
- If you’re running Windows, you’ll need to serve the application using IIS.
- If you’re running Linux, you’ll need to use Apache2 and won’t have the option of using a MSSQL database.
Full installation and configuration documentation is provided for both options:
For many customers, it’s a lot faster and simpler to use Docker containers for the purpose of orchestrating all the various application layers & components. Instructions for using our TestRail Docker image can be found here.
Future Deprecation Plans
From time to time, based on customer feedback or other operational factors, we may decide to remove, retire or replace elements of the TestRail architecture and functionality.
Based on customer feedback and our own operational experiences, the additional Cassandra component currently in use for the purpose of storing attachment metadata, is targeted for deprecation soon. We don’t have a precise timeline for the removal currently, but we are actively working on the alternative architecture to support it.
Cloud customers should largely be unaffected by this change, since the management of attachments occurs behind the scenes. Server customers will benefit from the removal of the additional Cassandra component, since we anticipate a much smoother and easier installation and support experience as a result.
Please watch this space for further information and updates.