TestRail review

I hope you all know that testing implies specific documentation, too. It can be run in an old school way, in xls-files and folders, or the way a real testing Jedi does - with a help of a special tool.

One of such tools is what I’d like to inform you about — the TestRail  we’ve implemented in TestLab² recently. The tool was found so good that I could not help sharing something as useful as this with you.

    Short official summary:
  • Official website: http://www.gurock.com/testrail/
  • Purpose: Maintaining testing documentation
  • License: Paid product, from €159
  • Status: release 1.0
  • Platform: LAMP

TestRail screenshot

For all the projects we process testing documentation is obligatory, from short checklist to branchy test suite with cross-links and hundreds of test cases. The idea to store documentation in local files was rejected right away, for it’s inefficient and simply unprofessional. We tried a “universal” solution, wiki (DokuWiki), but it wouldn’t take on. It’s hard to tell why, perhaps with the lapse of time we could have got used to it, but the TestRail appeared, so wiki-sort stuff was forgotten immediately.

    Short terminology disclaimer:
  • Test case — a test-plans’ atomic element, consists of following component.
  • Case run environment description (it can be moved to a higher level – to the test-plan)
  • Steps to reproduce
  • Expected result
  • Test suite is a list of cases, combined according to any common factor (whole product, specific feature, etc.)
  • Test plan is a list of test suites, chosen for the test
  • Test run – is a run of the test plan

Installation and System Requirements

Two variants: typical LAMP or “IIS+SQLServer+FastCGI/PHP”. The ionCube and cURL is also needed, but it’s not uncommon. It works rather fast, and after enabling gzip compression using mod_deflate it started to run at railway speed.

Basic features

Multiuser multiproject system for testing documentation maintaining and tests results accounting. The system provides jobsite for everyone: for test managers, who are interested in “when it’s to be done” and “how many bugs are found”; for ordinary testers, who find easy way to report completed cases; for analysts, who’re trying to crack the system behind app issues and understand what’s causing the troubles.

Workflow

TestRail screenshot

  1. Create a project, point out “Defect View Url” and “Defect Add Url”. The integration with bug-tracker is simple and yet brilliant: http://your-bug-tracker.com/view/%id%, and TestRail automatically substitutes this bug %id%. It fits the majority of bug-trackers.
  2. Create Milestone: actually, it is a version of a product tested and a deadline.
  3. Create a Test Suite:a set of test cases, e.g. “UI testing”.
  4. Within a Test Suite cases and sections of cases are created: for example, in “UI testing” there are two sections: “Client UI” and “Admin dashboard UI”. These sections support unlimited nesting.
  5. Each specific test case consists of Preconditions, Steps and Expected Result. I presume, everything is obvious here. You can attach pictures to them and cross-link them.
  6. After describing the testing system with the means of test cases, it is time to start testing. You need to create a Test Run, which includes necessary Test Suite or specific test cases. An executing person (from a list of active system users) is assigned to it, and it’s the final point for a test designer.
  7. A tester starts executing a test-run as he receives a notification about it. After tests completion each test case receives a status according to test results: Passed, Failed or Retest. Comments and found bugs’ IDs are also posted there.
    After the completion of all tests, the test run is closed, results are sent to the Task Owner, and all data about tests is kept in the system.
  8. Optionally, all the results can be exported to different formats, printed or even dumped to xml for some special use. TestRail creates some graphs for quick situation and application quality analysis.

Competitors comparison

I’ve exeperience working with two alternative systems: Rational Quality Manager and TestLink.  I beg open-source upholders’ pardon, but never I want to deal with the TestLink again. It is free, and that’s a telling argument, but that’s the only merit. Interface is old and inefficient; the system of cases and suites linking is poorly thought-out; the structure of suites… you know, in my (and entire teams’) opinion – “absolutely ineligible”.

The entire Rational Suite is great, but it’s too massive for small projects, and the propagation of its own methodology is overdosed. Let alone the price, which is comparable to the price of a batch of magic carpets.

Also I’d like to mention Bugzilla add-on named Testopia, but I haven’t seen it on stream myself and never heard of someone using it in production. If you use Bugzilla in your project, I think you need to pay attention to Testopia, because the high level of integration is promised. Also, there are some similar solutions, inbuilt to integrated development systems (i.e. Mercury) or desktop standalone products (TestLog, QaTraq). They don’t fit us because of lack of normal web-access.

Personal impression

The system proved to be almost perfect for us, both for its’ features and everyday usability. We’re involved in outsourced testing and run lots of small projects, and because of that the vendors’ license policy really fits us: the limit is applied to active system users only, so we deactivate client accounts at the end of the project.

Only two shortcomings are to be mentioned: current version of TestRail doesn’t support Unicode, which means, that it’s impossible to maintain documentation in Russian/Ukrainian without any hacks. The respective feature request is already announced. It is not an essential feature for our team, as all projects deserving to be maintained in TestRail are in English. The second problem is the lack of any WYSIWYG-editor in all text edit fields. It makes engineers happy, but regularly makes our customers feel confused.


Summary

If you maintain testing documentation of a big project or get lost in tons of doc- and xls- files with cases, you need to pay attention to TestRail. This product is not something everyone needs, but if you’re professional in testing, remember the name, and maybe TestRail will come in useful to you when processing some further projects.


Comments

  1. http://www.online—-casino.org/keno keno [url=http://www.online—-casino.org/keno] keno [/url] http://www.online—-casinos.info/alphanum_online online [url=http://www.online—-casinos.info/alphanum_online] online [/url] http://www.online—-casinos.info/slots slots [url=http://www.online—-casinos.info/slots] slots [/url] http://www.online—-casinos.info/blackjack blackjack [url=http://www.online—-casinos.info/blackjack] blackjack [/url] http://www.online—-casinos.info/roulette roulette [url=http://www.online—-casinos.info/roulette] roulette [/url] http://www.online—-casinos.info/poker poker [url=http://www.online—-casinos.info/poker] poker [/url] http://www.online—-casinos.info/craps craps [url=http://www.online—-casinos.info/craps] craps [/url] http://www.online—-casinos.info/baccarat baccarat [url=http://www.online—-casinos.info/baccarat] baccarat [/url] http://www.online—-casinos.info/keno keno [url=http://www.online—-casinos.info/keno] keno [/url] 96820@gmail.com

    online slots on
  2. http://www.erotismoboutique.com/ultram.html cheap ultram online [url=http://www.erotismoboutique.com/ultram.html] cheap ultram online [/url] http://www.erotismoboutique.com/valtrex.html cheap valtrex online [url=http://www.erotismoboutique.com/valtrex.html] cheap valtrex online [/url] http://www.erotismoboutique.com/viagra.html cheap viagra online [url=http://www.erotismoboutique.com/viagra.html] cheap viagra online [/url] http://www.erotismoboutique.com/zimulti.html cheap zimulti online [url=http://www.erotismoboutique.com/zimulti.html] cheap zimulti online [/url] http://www.erotismoboutique.com/xenical.html cheap xenical online [url=http://www.erotismoboutique.com/xenical.html] cheap xenical online [/url] http://www.erotismoboutique.com/zithromax.html cheap zithromax online [url=http://www.erotismoboutique.com/zithromax.html] cheap zithromax online [/url] http://www.erotismoboutique.com/zoloft.html cheap zoloft online [url=http://www.erotismoboutique.com/zoloft.html] cheap zoloft online [/url] http://www.erotismoboutique.com/zovirax.html cheap zovirax online [url=http://www.erotismoboutique.com/zovirax.html] cheap zovirax online [/url] http://www.erotismoboutique.com/zyrtec.html cheap zyrtec online [url=http://www.erotismoboutique.com/zyrtec.html] cheap zyrtec online [/url] http://www.teservicesinc.com/ levitra online [url=http://www.teservicesinc.com/] levitra online [/url] 51207@gmail.com

    motorcycle insurance rate on
  3. http://www.rochestergentledental.com/ health insurance plan [url=http://www.rochestergentledental.com/] health insurance plan [/url] http://www.rochestergentledental.com/ health insurance plans [url=http://www.rochestergentledental.com/] health insurance plans [/url] http://www.rochestergentledental.com/ health insurance providers [url=http://www.rochestergentledental.com/] health insurance providers [/url] http://www.rochestergentledental.com/ health insurances [url=http://www.rochestergentledental.com/] health insurances [/url] http://www.rochestergentledental.com/ individual health insurance quotes [url=http://www.rochestergentledental.com/] individual health insurance quotes [/url] http://www.rochestergentledental.com/ low cost health insurance [url=http://www.rochestergentledental.com/] low cost health insurance [/url] http://www.rochestergentledental.com/ health insurance cost [url=http://www.rochestergentledental.com/] health insurance cost [/url] http://www.rochestergentledental.com/ discount health insurance [url=http://www.rochestergentledental.com/] discount health insurance [/url] http://www.rochestergentledental.com/ health insurance coverage [url=http://www.rochestergentledental.com/] health insurance coverage [/url] http://www.teservicesinc.com/ levitra online [url=http://www.teservicesinc.com/] levitra online [/url] 59911@gmail.com

    motorcycle insurance companies on
  4. http://www.levitra-online.biz/item/erectile_dysfunction/brand_viagra.html Brand Viagra [url=http://www.levitra-online.biz/item/erectile_dysfunction/brand_viagra.html] Brand Viagra [/url] http://www.levitra-online.biz/item/erectile_dysfunction/levitra_super_active.html Levitra Super Active+ [url=http://www.levitra-online.biz/item/erectile_dysfunction/levitra_super_active.html] Levitra Super Active+ [/url] http://www.levitra-online.biz/item/erectile_dysfunction/apcalis_sx.html Apcalis SX [url=http://www.levitra-online.biz/item/erectile_dysfunction/apcalis_sx.html] Apcalis SX [/url] http://www.levitra-online.biz/item/erectile_dysfunction/sublingual_cialis.html Sublingual Cialis [url=http://www.levitra-online.biz/item/erectile_dysfunction/sublingual_cialis.html] Sublingual Cialis [/url] http://www.levitra-online.biz/item/erectile_dysfunction/yagara.html Yagara [url=http://www.levitra-online.biz/item/erectile_dysfunction/yagara.html] Yagara [/url] http://www.levitra-online.biz/item/erectile_dysfunction/sublingual_viagra.html Sublingual Viagra [url=http://www.levitra-online.biz/item/erectile_dysfunction/sublingual_viagra.html] Sublingual Viagra [/url] http://www.buy—viagra-online.com/item/erectile_dysfunction/viagra.html Viagra [url=http://www.buy—viagra-online.com/item/erectile_dysfunction/viagra.html] Viagra [/url] http://www.buy—viagra-online.com/item/erectile_dysfunction/cialis.html Cialis [url=http://www.buy—viagra-online.com/item/erectile_dysfunction/cialis.html] Cialis [/url] http://www.buy—viagra-online.com/item/erectile_dysfunction/levitra.html Levitra [url=http://www.buy—viagra-online.com/item/erectile_dysfunction/levitra.html] Levitra [/url] 99014@gmail.com

    Free Insurance Quotes on
  5. http://www.cheap—cialis.info/item/erectile_dysfunction/brand_cialis.html Brand Cialis [url=http://www.cheap—cialis.info/item/erectile_dysfunction/brand_cialis.html] Brand Cialis [/url] http://www.cheap—cialis.info/item/erectile_dysfunction/brand_viagra.html Brand Viagra [url=http://www.cheap—cialis.info/item/erectile_dysfunction/brand_viagra.html] Brand Viagra [/url] http://www.cheap—cialis.info/item/erectile_dysfunction/levitra_super_active.html Levitra Super Active+ [url=http://www.cheap—cialis.info/item/erectile_dysfunction/levitra_super_active.html] Levitra Super Active+ [/url] http://www.cheap—cialis.info/item/erectile_dysfunction/apcalis_sx.html Apcalis SX [url=http://www.cheap—cialis.info/item/erectile_dysfunction/apcalis_sx.html] Apcalis SX [/url] http://www.cheap—cialis.info/item/erectile_dysfunction/sublingual_cialis.html Sublingual Cialis [url=http://www.cheap—cialis.info/item/erectile_dysfunction/sublingual_cialis.html] Sublingual Cialis [/url] http://www.cheap—cialis.info/item/erectile_dysfunction/yagara.html Yagara [url=http://www.cheap—cialis.info/item/erectile_dysfunction/yagara.html] Yagara [/url] http://www.cheap—cialis.info/item/erectile_dysfunction/sublingual_viagra.html Sublingual Viagra [url=http://www.cheap—cialis.info/item/erectile_dysfunction/sublingual_viagra.html] Sublingual Viagra [/url] http://www.buy—viagra.us/item/erectile_dysfunction/viagra.html Viagra [url=http://www.buy—viagra.us/item/erectile_dysfunction/viagra.html] Viagra [/url] http://www.buy—viagra.us/item/erectile_dysfunction/cialis.html Cialis [url=http://www.buy—viagra.us/item/erectile_dysfunction/cialis.html] Cialis [/url] 61370@gmail.com

    Free Insurance Quotes on
  6. http://www.life—-insurance.us/allcompanies Life Insurance [url=http://www.life—-insurance.us/allcompanies] Life Insurance [/url] http://www.life—-insurance.us/auto Life Insurance [url=http://www.life—-insurance.us/auto] Life Insurance [/url] http://www.life—-insurance.us/health Life Insurance [url=http://www.life—-insurance.us/health] Life Insurance [/url] http://www.life—-insurance.us/home Life Insurance [url=http://www.life—-insurance.us/home] Life Insurance [/url] http://www.life—-insurance.us/life Life Insurance [url=http://www.life—-insurance.us/life] Life Insurance [/url] http://www.life—-insurance.us/other Life Insurance [url=http://www.life—-insurance.us/other] Life Insurance [/url] http://www.private—health—insurance.com/allcompanies Private Health Insurance [url=http://www.private—health—insurance.com/allcompanies] Private Health Insurance [/url] http://www.private—health—insurance.com/auto Private Health Insurance [url=http://www.private—health—insurance.com/auto] Private Health Insurance [/url] http://www.private—health—insurance.com/health Private Health Insurance [url=http://www.private—health—insurance.com/health] Private Health Insurance [/url] 82971@gmail.com

    online casinos on
  7. http://www.viagra—-online.com/item/erectile_dysfunction/brand_cialis.html Brand Cialis [url=http://www.viagra—-online.com/item/erectile_dysfunction/brand_cialis.html] Brand Cialis [/url] http://www.viagra—-online.com/item/erectile_dysfunction/brand_viagra.html Brand Viagra [url=http://www.viagra—-online.com/item/erectile_dysfunction/brand_viagra.html] Brand Viagra [/url] http://www.viagra—-online.com/item/erectile_dysfunction/levitra_super_active.html Levitra Super Active+ [url=http://www.viagra—-online.com/item/erectile_dysfunction/levitra_super_active.html] Levitra Super Active+ [/url] http://www.viagra—-online.com/item/erectile_dysfunction/apcalis_sx.html Apcalis SX [url=http://www.viagra—-online.com/item/erectile_dysfunction/apcalis_sx.html] Apcalis SX [/url] http://www.viagra—-online.com/item/erectile_dysfunction/sublingual_cialis.html Sublingual Cialis [url=http://www.viagra—-online.com/item/erectile_dysfunction/sublingual_cialis.html] Sublingual Cialis [/url] http://www.viagra—-online.com/item/erectile_dysfunction/yagara.html Yagara [url=http://www.viagra—-online.com/item/erectile_dysfunction/yagara.html] Yagara [/url] http://www.viagra—-online.com/item/erectile_dysfunction/sublingual_viagra.html Sublingual Viagra [url=http://www.viagra—-online.com/item/erectile_dysfunction/sublingual_viagra.html] Sublingual Viagra [/url] http://www.levitra—online.com/item/erectile_dysfunction/viagra.html Viagra [url=http://www.levitra—online.com/item/erectile_dysfunction/viagra.html] Viagra [/url] http://www.levitra—online.com/item/erectile_dysfunction/cialis.html Cialis [url=http://www.levitra—online.com/item/erectile_dysfunction/cialis.html] Cialis [/url] 60540@gmail.com

    buy cialis on
  8. http://www.artfangled.com/ homeowner insurance [url=http://www.artfangled.com/] homeowner insurance [/url] http://www.artfangled.com/ homeowners insurance quotes [url=http://www.artfangled.com/] homeowners insurance quotes [/url] http://www.artfangled.com/ homeowners insurance [url=http://www.artfangled.com/] homeowners insurance [/url] http://www.artfangled.com/ affordable home insurance [url=http://www.artfangled.com/] affordable home insurance [/url] http://www.bedroomchefapron.com/ cheap motorcycle insurance [url=http://www.bedroomchefapron.com/] cheap motorcycle insurance [/url] http://www.bedroomchefapron.com/ motorcycle insurance quotes [url=http://www.bedroomchefapron.com/] motorcycle insurance quotes [/url] http://www.bedroomchefapron.com/ motorcycle insurance [url=http://www.bedroomchefapron.com/] motorcycle insurance [/url] http://www.bedroomchefapron.com/ motorcycle life insurance [url=http://www.bedroomchefapron.com/] motorcycle life insurance [/url] http://www.bedroomchefapron.com/ motorcycle insurance broker [url=http://www.bedroomchefapron.com/] motorcycle insurance broker [/url] http://www.teservicesinc.com/ levitra online [url=http://www.teservicesinc.com/] levitra online [/url] 88280@gmail.com

    small business insurance on