Six months back, a testing professional setup a call with me and asked “I just took up this testing role. I have inherited plenty of automation code. There are no testcase lists. But I don’t want to use automation, and I want to test everything by humans. What is the approach I should take?”
It was interesting, because we usually hear stories of test cases getting converted to automation scripts, but in this case, it was the opposite. I didn’t ask that person why they wanted to do that because I knew from what they had shared so far that they had difficulty understanding the product as well as the automation scripts, but of course, not ready too to go through the automation scripts and learn what’s going on, probably because the automation scripts were outdated!
With testcases and automation scripts, we face this issue every day. There have been some approaches for not having test cases, and those approaches align with exploration, learning, test designing on the fly, and interpreting the test results. Two days back, I spoke with a practitioner who does exploratory testing as part of their daily work, and they shared that there are practical difficulties in not having test cases. Exploratory testing lets us take notes after each session, and I pointed that out to them. But they said that’s not sufficient level of detail for someone entering the team who want to learn the product from scratch. It was not clear to me why someone entering the team new would not look at the product firsthand and why they are looking at the test cases. I presume it was because of the time crunch and not having sufficient time to explore by themselves and they needed to rely on the test cases.
The other usage why test cases existed, they said, was for converting them to automated checks wherever possible.
I know Testing purists would shake their heads in disbelief for both of these reasons. Ideally, automation checks should be part of testing efforts done by humans and should not be separated out. In fact, exploratory testing sessions should also use automation as part of their checks that they want to do while exploring. But in reality, testing is divided into checks that can be automated and exploration to find unusual conditions and corner case defects. And this disconnect is the reason for several headaches in test scenarios.
I had an interesting discussion sometime back where I argued that regression testing cannot be separated out from exploratory testing because regression testing should not be looked as an opportunity for testcase automation, but exploration should be present in it such that the effect of the defect fix is thoroughly tested as a combination of human effort and automation.
So, should we write test cases or not? It depends. There are many reasons. One, if the product details (like is GUI) are too involved, the note-taking itself will be detailed. So the team might as well write test cases. The second reason is that it would depend on the person who is doing the testing – how effective they are in testing, making relevant notes, etc. If the person is adept, then their testing and their notes would provide enough information for anyone who is looking at the product. If they aren’t, the team is better off with test cases. Finally, regulatory and compliance requirements of some projects will require test cases.
What are your thoughts? Share them in the comments below. Feel free to contact me to discuss about testcase decisions.