Software Testing is a field that has a wealth of opportunities to explore deeper on how to test software systems well. There are different approaches to come up with a test strategy, and risk-based testing is one of them. The words ‘risk’ and ‘investigation’ have been there for quite a long time, getting discussed in social media and testing conferences, when it comes to Software Testing.
Let’s look at risk-based testing. I found this well written article on risk-based testing, and it prompted me to let my thoughts flow. To summarize, that article concluded that three things that they think are important for risk-based analysis are (a) impact, (b) chance of failure, and (c) frequency of usage or occurrence. That’s a good summary of how to go about the analysis. I did a Twitter chat with the author, and in the process, thought about the pros and cons of risk-based testing.
The author’s analysis is fine as far as risks assessment of individual test cases or test scenarios are concerned, which provides a good blueprint for assigning priority for test scenarios that need to be handled. It is something akin to unit tests, where risks are handled at their ‘module’ level scope. My major consideration on this is to do with Risk Value Assignment.
While prioritizing test cases based on risk, it becomes pertinent to come up with a ‘risk value’ for that test scenario/test case, so that it can be prioritized. We tend to normalize risk values, and this becomes a mundane task. Then we get tempted to automate the risk value assignment through RPA or Machine Learning, so that we can focus on more ‘important’ things. Then we would have a problem at hand, because risk assessment and prioritization is effective only when it is done by humans. But people won’t pay attention to this fact, as they are in a hurry to automate everything.
The next problem I have with prioritizing is that the risk value assignment is localized and concerned with individual test scenarios or test cases. More often than not, big outages in production happen when multiple software features interact with each other in unprecedented ways. Usually, a test scenario that’s perceived as low/very low risk interacts with another feature and ends up as a big production problem. In a complex solution, many such things happen, so risk prioritization to individual test cases is insufficient. We need to come up with a cumulative risk value based on the interactions of these features for various scenarios.
Even if we come up with risk values for complex solution testing scenarios, the chances of those risk values being treated as a misused, misinterpreted metric and dashboard-ed is high in the normal, mundane, high-speed course of business, rather than risk value being used as a guiding factor to come up with more effective testing scenarios. In this case, it’s rather worthwhile to spend the time directly coming up with better test scenarios than focusing on risk value assessment and assignment.
As a Software Testing practitioner, one should still strive to come up with prioritizing risks on solution level scenarios in addition to individual test scenarios, in order for this effort to reap benefits on catching those nasty defects, but beware that the effectiveness of coming up with the scenarios and assigning them proper priorities is entirely left to the competence of the practitioner in question.
If you need professional guidance on how to arrive at risk prioritization for your project in your organisation, feel free to schedule a time with me to discuss. Happy Testing!
