TestRail review
I hope you all know that testing implies specific documentation, too. It can be run in an old school way, in xls-files and folders, or the way a real testing Jedi does - with a help of a special tool.
One of such tools is what I’d like to inform you about — the TestRail we’ve implemented in TestLab² recently. The tool was found so good that I could not help sharing something as useful as this with you.
- Short official summary:
- Official website: http://www.gurock.com/testrail/
- Purpose: Maintaining testing documentation
- License: Paid product, from €159
- Status: release 1.0
- Platform: LAMP
For all the projects we process testing documentation is obligatory, from short checklist to branchy test suite with cross-links and hundreds of test cases. The idea to store documentation in local files was rejected right away, for it’s inefficient and simply unprofessional. We tried a “universal” solution, wiki (DokuWiki), but it wouldn’t take on. It’s hard to tell why, perhaps with the lapse of time we could have got used to it, but the TestRail appeared, so wiki-sort stuff was forgotten immediately.
- Short terminology disclaimer:
- Test case — a test-plans’ atomic element, consists of following component.
- Case run environment description (it can be moved to a higher level – to the test-plan)
- Steps to reproduce
- Expected result
- Test suite is a list of cases, combined according to any common factor (whole product, specific feature, etc.)
- Test plan is a list of test suites, chosen for the test
- Test run – is a run of the test plan
Installation and System Requirements
Two variants: typical LAMP or “IIS+SQLServer+FastCGI/PHP”. The ionCube and cURL is also needed, but it’s not uncommon. It works rather fast, and after enabling gzip compression using mod_deflate it started to run at railway speed.
Basic features
Multiuser multiproject system for testing documentation maintaining and tests results accounting. The system provides jobsite for everyone: for test managers, who are interested in “when it’s to be done” and “how many bugs are found”; for ordinary testers, who find easy way to report completed cases; for analysts, who’re trying to crack the system behind app issues and understand what’s causing the troubles.
Workflow
- Create a project, point out “Defect View Url” and “Defect Add Url”. The integration with bug-tracker is simple and yet brilliant: http://your-bug-tracker.com/view/%id%, and TestRail automatically substitutes this bug %id%. It fits the majority of bug-trackers.
- Create Milestone: actually, it is a version of a product tested and a deadline.
- Create a Test Suite:a set of test cases, e.g. “UI testing”.
- Within a Test Suite cases and sections of cases are created: for example, in “UI testing” there are two sections: “Client UI” and “Admin dashboard UI”. These sections support unlimited nesting.
- Each specific test case consists of Preconditions, Steps and Expected Result. I presume, everything is obvious here. You can attach pictures to them and cross-link them.
- After describing the testing system with the means of test cases, it is time to start testing. You need to create a Test Run, which includes necessary Test Suite or specific test cases. An executing person (from a list of active system users) is assigned to it, and it’s the final point for a test designer.
- A tester starts executing a test-run as he receives a notification about it. After tests completion each test case receives a status according to test results: Passed, Failed or Retest. Comments and found bugs’ IDs are also posted there.
After the completion of all tests, the test run is closed, results are sent to the Task Owner, and all data about tests is kept in the system. - Optionally, all the results can be exported to different formats, printed or even dumped to xml for some special use. TestRail creates some graphs for quick situation and application quality analysis.
Competitors comparison
I’ve exeperience working with two alternative systems: Rational Quality Manager and TestLink. I beg open-source upholders’ pardon, but never I want to deal with the TestLink again. It is free, and that’s a telling argument, but that’s the only merit. Interface is old and inefficient; the system of cases and suites linking is poorly thought-out; the structure of suites… you know, in my (and entire teams’) opinion – “absolutely ineligible”.
The entire Rational Suite is great, but it’s too massive for small projects, and the propagation of its own methodology is overdosed. Let alone the price, which is comparable to the price of a batch of magic carpets.
Also I’d like to mention Bugzilla add-on named Testopia, but I haven’t seen it on stream myself and never heard of someone using it in production. If you use Bugzilla in your project, I think you need to pay attention to Testopia, because the high level of integration is promised. Also, there are some similar solutions, inbuilt to integrated development systems (i.e. Mercury) or desktop standalone products (TestLog, QaTraq). They don’t fit us because of lack of normal web-access.
Personal impression
The system proved to be almost perfect for us, both for its’ features and everyday usability. We’re involved in outsourced testing and run lots of small projects, and because of that the vendors’ license policy really fits us: the limit is applied to active system users only, so we deactivate client accounts at the end of the project.
Only two shortcomings are to be mentioned: current version of TestRail doesn’t support Unicode, which means, that it’s impossible to maintain documentation in Russian/Ukrainian without any hacks. The respective feature request is already announced. It is not an essential feature for our team, as all projects deserving to be maintained in TestRail are in English. The second problem is the lack of any WYSIWYG-editor in all text edit fields. It makes engineers happy, but regularly makes our customers feel confused.
Summary
If you maintain testing documentation of a big project or get lost in tons of doc- and xls- files with cases, you need to pay attention to TestRail. This product is not something everyone needs, but if you’re professional in testing, remember the name, and maybe TestRail will come in useful to you when processing some further projects.
Comments
- online slots on February 15, 2010, at 04:59 AM
- motorcycle insurance rate on February 15, 2010, at 10:06 AM
- motorcycle insurance companies on February 22, 2010, at 01:28 AM
- Free Insurance Quotes on February 22, 2010, at 07:33 PM
- Free Insurance Quotes on February 24, 2010, at 06:06 AM
- online casinos on February 24, 2010, at 06:13 PM
- buy cialis on February 27, 2010, at 11:44 PM
- small business insurance on March 3, 2010, at 07:50 AM