In a world of cloud-based tooling, sometimes what you really need is to see how your app might work on a real device. I have visited this topic in a previous post, when I discussed the factors you should take into account when conducting cross-browser testing.
I’d like to zoom in closer on the last point (real devices vs emulators) and talk about how this applies in a test environment. More specifically: how can using a test lab of real devices help you to extend your testing activities and maximise potential test coverage?
Firstly, what is the goal of a test lab?
A test lab is a place where software testers conduct controlled experiments with known measurements in a controlled environment. The aim is to mimic a live production environment as closely as possible.
There is a whole range of fantastic resources out there that will teach how to build your own test lab, such as this one written by Destiny Montague and Lara Hogan. And building a test lab doesn’t have to be a mammoth task, especially if you only have a few requirements. But if you are unsure about where to start, or have short-term testing requirements, an external test or device lab that has already been built is your new best friend.
These are some ways using a readily-available device lab can be more advantageous than building one:
1. Increase the range of possible configurations that you may not have yourself.
Test labs like ours offer the opportunity to test your app on a wider range of possible configurations that you may not have yourself, because we work with a large client base with differing device requirements. Because of this, we can offer a full range of platforms and configurations that you do not have to source beforehand.
You can thus increase your test coverage, not leave a certain configuration out because it is too expensive, and minimise the risk of a certain segment of your user base finding bugs in your application. Plus, there is no need to maintain your test lab with the latest releases, as QA practitioners also keep a detailed and up-to-date list of possible configurations that clients may need, and will most likely already have the devices to hand.
2. You don’t have to spend time shopping for cost-effective devices and then maintaining them.
Choosing the devices that you will include in your test lab involves deciding upon which configurations will be enough to cover as much of your user base as possible. You then have to consider the cost. And costs can escalate pretty quickly if there isn’t proper planning in place, which can then obviously hinder when testing will take place. Using an external party’s test lab means you can start your testing earlier, on devices that have already been purchased and audited for use. It’s also more cost-effective, as you’d only pay for the hourly rate of the tester – with the rent of the test lab included – instead of paying for the initial cost and regular upkeep of your own set of devices.
3. Greater chance of keeping a testing environment stable.
Using a device lab that has already been built, especially if it has been built by a specialised software testing group, means that it will have been developed in accordance with proper QA knowledge and methodology. There is then less of a chance that the testing environment will be contaminated, especially if version control is taken into account.
Version control is any kind of practice that tracks and provides changes to source code. Because bugs often only occur in certain versions of the same piece of software, it is vital that in the process of locating and fixing them, testers and developers need to be able to retrieve and run different versions to determine where the issue lies. A functioning test lab will provide you with tools that will help you trace the history of changes to your test devices, thereby minimising mistakes and keeping your test environment more stable.
4. You don’t have to think about the logistical issues.
As well as choosing the software and hardware you need to get a device lab up-and-running, and the cost, you’ll need to think about where the device lab will go. You’ll need to ask yourself questions such as:
Quite simply, using an external test lab means this work is already done for you. The owners will be the ones storing the devices and making sure the software and hardware are properly updated and ready to go. The logistics of setting up a lab are already catered for, so you can carry out your testing much quicker.
Minimise the risk of bugs appearing when your app goes into production by increasing your cross-browser test coverage. We have a test lab with a full, up-to-date range of devices and platforms available to help you do this. Get in contact with us to discuss your requirements, and together we’ll create a tailor-made testing plan for your app, on our technology.