I really wish there’s an encyclopedia for Software Testing so that people don’t ask the same questions, again, and again, and again. I don’t need to explicitly say that because if you go through websites like Reddit, Quora, or even LinkedIn groups, you will find the same questions being repeated in spite of them being answered already. One such question is ‘What is the best testing metric?’ Let’s talk about this in today’s blog.
What is a metric? It’s a means to measure something quantitatively. Why do you want to measure something? Because you want that number to indicate to you in some way if you are being successful in achieving what you targeted to reach.
If the above is clear enough to you, you would not go to someone else to ask ‘What is the best testing metric?’ Because what success is for you is entirely up to you. If I go to a sprint runner and tell them that their best measure of success would be the number of goals they scored, how would that be? It is the same thing to try to advice some team on what their best testing metric would be without looking at their present situation, where they want to get, and how they want to get there!
If you are a process/quality consultant, you already know the answer. You ask more and more questions to the person asking for that metric to figure out about their scenario, rather than blurting out an answer like ‘defect-fix-time’. Yes, there are some metrics that might have worked well in some contexts but who knows what would work for your team when there are umpteen number of contexts like Scrum, Kanban, XP, Scrumban, Kanscrum, …. (do they have ScruXP?)
Also, metrics have different purposes. Test leads use a set of metrics. Automation engineers use some other. Test managers use something else. Test organisation leaders use something else. Without knowing who wants to use the metric, it is not possible to suggest one.
Also remember that test metric could vary from release to release of the same product, depending on how much success you have achieved on a specific objective and whether you want to measure it anymore!
So, the best answer is as the consultants always say: ‘It depends’. They are right. And it makes sense.
For looking at your team’s current situation and come up with metrics, feel free to contact me.