Here at spriteCloud HQ, we all share the same background story. Details vary wildly, of course, but some or all of the following holds true for each one of us:
We’ve all worked at companies where software quality assurance didn’t exist. We’ve worked at companies where it didn’t exist, yet some people maintained it did. We’ve worked at companies where SQA was an afterthought, and consequently failed. We’ve worked at companies where that was taken as a reason not to bother with SQA. We’ve worked at companies where people didn’t even know what SQA was. We’ve worked at companies who said “our developers can do QA”, or “we get free QA from our beta users”, and whose software was then ripped apart by users for its bad performance or lack of stability.
We’ve also worked at companies where SQA was taken seriously and executed well, but for most of us those were few and far between.
That begs the question: why is that the case?
More precisely, why is it that a multi-billion dollar industry survives and thrives when it systematically neglects an aspect of product development that no other industry can afford to neglect?
Now let me qualify that previous statement: the big players, the likes of Amazon, Google, Facebook, etc. — they take SQA seriously. I would go as far and say that they are amongst the most powerful drivers of innovation in the field of SQA, both in terms of tools and processes.
With that said, maybe the above question needs to be rephrased: what is it that prevents so many smaller software forges from adopting SQA (properly)?
In obvious answer to that, that these companies lack experience with SQA, is also trite. Experience can be hired.
The real problem appears to be that SQA is not the clearly definable step in the software development life cycle that it appears to be at a glance. You’re all familiar with the waterfall model for software development, and with agile counterparts that iterate over most of the same steps indefinitely.
It’s easy to assume that one of the steps in that famous waterfall model, “verification”, is what SQA is — and that consequently it should happen after (an iteration of) development is done, and that’s the beginning and end of it.
Nothing could be further from the truth.
Properly conducted SQA is not so much verification of your product’s functionality, but verification that your software development process yields high quality at each step.
- Requirements need to be verified for their suitability to your business. All too often, product requirements appear “magically” out of thin air, maybe copied from some vaguely related competing product. Those aren’t a bad start, mind you — but for the next iteration of your software, these requirements need to change to adapt to what you’ve learned with the previous iteration. SQA can help in that by making the effects of requirements measurable, and comparing those metrics as requirements are adapted.
- Design needs to be verified; at Joost, back in the days when the company was providing a video desktop client, the button for sharing videos with your friends was changed from white to green in one release. Nothing else in the user interface changed. As a consequence, the number of users who used the button increased manifold. Providing metrics “before” and “after” (A/B testing) gives a sound basis for making design decisions.
But of course this is not limited to visual design alone; in a similar fashion, metrics can verify that a redesigned algorithm increases the performance of your software, for example.
- Implementation is the focus of most software testing, and what people usually think of as SQA. The primary focus here is to provide consistently repeatable tests in well-defined environments. Continuous integration provides one of the best methods for ensuring that software works as the developers intended.
- Verification encompasses not only testing the implementation, but goes beyond that and includes testing that the software works as designed (whether or not the design is good; that question is answered above).
- Maintenance is not a phase usually found in agile development models, as these methodologies tend to loop back to a new requirements phase once software is verified and published. Nonetheless with regards to SQA, a maintenance phase does exist and should exist in the form of regression testing: ensuring that future versions of the software do not break behaviour that was verified in previous versions.
As you can see, SQA touches on all areas of software development. And you can also see, that SQA is largely concerned with providing metrics in order to gain an insight into your product’s performance (in traditional quality management that’s closer to the realm of quality control).
The role of SQA, then, is not merely to ensure your product works. It is to ensure the business you are building around your product works the way you intended, by ensuring that your product’s target audience is satisfied.
It’s worth noting that because SQA and software product development are so closely intertwined, it follows that there is no silver bullet to SQA processes and tools, just like there is no silver bullet to a software development processes and tools. But as with development, there is a rich pool of techniques, technologies and experience to draw on.
And with that said, how can you afford to neglect SQA if you want your business to thrive?