metrics quality quality engineering software testing

Five Crimes against Testing

In the past metrics have often been the weapon of choice in crimes against testing.

Five crimes against testing that I see repeatedly executed.

Testing and Quality are the same things

Just because you test, it doesn’t mean you have a quality product.  Sounds obvious right? But why then is quality assigned to one person. Why have a Quality Gatekeeper? Testing shines a light on threats to quality.Testing does diddly squat if no developer fixes the problems, or no delivery manager allocates time for bug fixing and re-testing.

Testing is easy

Nope, testing is not easy. It might appear easy, and perhaps superficial testing is easy.  Testing is a skilled activity requiring critical thinking, a solid understanding of context (be that business or technical and typically both), great communication and a host of other skills and traits.

Testing can be 100% automated

No. See above. Yes, you can automate parts of the execution, but humans design and evaluate plus a host of other activities. Critical thinking and problem-solving skills are essential parts of testing. Without them, you might as well automate a fish on a bicycle.


In the past, metrics have been the weapon of choice in crimes against testing. Test Case counts, DRE (Defect Removal Efficiency), pass/fail rates have all contributed to poor testing behaviour. Worse, they’ve often inhibited change due to dependencies on these metrics. Many testers have been burned badly by these metrics to the point where it’s stilted the conversation around quality metrics. The crime here is to throw out metrics altogether. Instead, why not have an informed discussion on the merits of metrics and quality?

Testing is a means to an end, not the end

It’s easy to get caught up in discussions on the merits of testing and checking. But ask yourself how this debate is helping your organisation deliver a quality product.  Being right semantically is not the end result. Even Testing is not the end result. Your end result is a quality product. If you are a tester or quality advocate in your organisation, do you know what quality means in terms of business outcomes?

I bet you’ve seen a few crimes of your own. What would you add?

By Anne-Marie Charrett

Anne-Marie Charrett is an internationally recognized expert in software testing and quality engineering.
She keynotes at international conferences on the topic of Quality, Coaching, and Leadership.
Ex-Head of Engineering at Tyro Payments where she transitioned testers to a quality coaching model
Consultant on Quality Engineering, developer of the quality operating model. Invented and rolled out a consulting model for quality engineering.
Consulting across FinTech, Media, Government, Insurance, Banking & Telco Sectors
Creator and Lecturer of Enterprise Software Testing course at UTS Australia. Co-developed a coaching model aiming to transfer testing skill and know-how using the Socratic method.
B.Eng (Hons) Electronic Engineering (I really am a quality engineer!)
Based in Sydney, Australia works – internationally.

6 replies on “Five Crimes against Testing”

How did you miss that?
Once a bug has been found it becomes obvious, but until then it can be very obscure. We might have tested a feature in a hundred different ways, but if one bug slips through we will get that question. And the implied answer is: "Because you’re a bad tester".
We have to remember that not even the best tester with unlimited resources will find all the bugs in a product.

To expand on #1 I often rephrase "quality" to a more meaningful word called "learning". You actually want more learning and want it faster instead of talking about "quality". Quality, in my opinion is a full company and team-wide outcome based on hiring, budget, decisions, vendors, politics, customers, processes, frameworks and so on. The whole system in play makes quality thus testers are not controlling quality. The end product is an accumulation of everything brought together and the customer votes with their money in many cases.

Leave a Reply

Your email address will not be published. Required fields are marked *