The challenges that we have as Software Testing professionals with examining and assessing if a product is on its path to serve its customers are increasing day by day. In addition to the existing challenges that we have with fixed test data which I wrote about in my recent article, there’s another new type of test data, which is an incoming stream, as in data-driven systems.
In the data world, it is not the individual, single piece of data that we might be dealing with, but we should also deal with sets of data that comes in that are not planned or not stored anywhere, and they have to be processed the moment they come in. These are ‘online’ data, and accordingly, the logic of the program that processes the data has to handle these varying sets. And accordingly, the test strategy to test these data also has to be dealt with.
Can one then predict all possible combinations of data streams that can come in? The answer is no. In this case, how do we figure in advance, as software testers, as how the system should behave? There are some approaches available, one being looking for certain patterns of data. There are challenges with this kind of approach too, because we don’t know the magnitude of the data, just its pattern. The program has to scale up/scale down the magnitude of incoming data so that it can recognize the pattern properly. Here again, we need to test if that scaling up and scaling down is done correctly.
Again, there is no assurance that we will be able to detect patterns. Also, we would have no clue on how frequently we should be listening for the data in order to decipher patterns. These are open questions for the system under test to answer and accordingly the test strategy would vary.
Finally, the processing that is done by the system under test for changing input data could be really challenging, because every piece of incoming data in the stream can change the system state or the context. When there is a context change, the testing should be done with each context change to verify that the system behaves as expected.
The data-based systems pose unique challenges to Software Testing. They need a solid strategy, along with tactical ability to handle challenges in testing these kind of systems.
For devising test strategy for data-driven systems, feel free to get in touch with me.