A test practitioner bravely declared in Twitter that customers don’t care about the types of Software Testing. That was a while back, probably two years back. Coming from a background of detailed testing culture, which contains many types of testing from white box testing to solution testing, I was taken aback. Because, the companies that I worked for before had made the efforts in educating the customers about the nature and importance of the various types of testing, and Software Testers often got the opportunity to interact with clients to get an idea of how they would use the product.
Fast forward some years, I look at the situation right now where apps fill our lives. The apps are not just stand-alone, but interact with various other apps, and that reinforces the importance of doing a variety of tests. In the engineering standards world, we have industry standards as RFCs and technology-specific standards like DOCSIS, which specify how products should behave. Unfortunately, today’s apps are free-for-all in terms of their behavior, and every engineer is an unicorn building an app. Even though there are customer ratings in places like Google Playstore to decide on which app we would need and use, the behavior and the quality of these apps are not fully validated and ascertained by an independent body and certified. No one knows how much testing has been done and what types of testing were done on these apps.
Yesterday, Ben Simo shared an experience in Twitter when he tried to pay at a restaurant and the payment failed. I was curious about why that happened and asked, and he replied with a couple of screenshots. In that particular scenario, I could figure that there are multiple components involved that interact with each other – the mobile phone, the password manager, the restaurant’s QR code and its software, the payment gateway, the payee’s bank, and maybe more. We wouldn’t exactly know what went wrong in the whole scheme of things unless we look into it closely, but when you are paying a restaurant for the food you eat, would you care to debug – even if you are tester?! In short, why should the customer do the testing for the apps?
These kind of problems have become commonplace now. In the urgency to release the products to the market, companies do very little testing in terms of functionality and get them released without doing solution testing with the various components involved and the various combinations. This has become a headache, and affects our daily lives in a significant way. Testing leaders regularly write in LinkedIn and Twitter about the problems they face with the apps that they use.
The companies claim that they don’t need to test proactively and can fix it later when the issue is encountered. Some developer practitioners claim that since the product is in Continuous Deployment, it can be fixed quickly when it is seen. Wrong. The idea is not to wait till the problem appears, because it would have affected people’s lives already. The idea is to find them by solution testing beforehand and fix them BEFORE the product reaches the customer.
The testing has to be done elaboratively and thinking about various angles, which is a human effort. Running unit and integration checks, and doing automation checks on the product is not sufficient. Certainly post-development testing is needed, and this testing is not just on the pipeline but should be done by humans with automation assisting them. I hope organisations take note of this important fact and solution test the product, as the ubiquitous, varied types of apps that access distributed services, databases, and data centers fill our lives on a daily basis.
For consulting on an effective solution test strategy, you can contact me.