Repeatability, and the ability to quickly come up with GUI screens have been a priority in application development for a while now. Applications that are called “no code” and “low code” generate applications so that one does not need to write applications from scratch every time. This is especially useful for startups who need to come up with quick front-end GUI and ability to revise their GUI based on the customer feedback. Reduced code is reduced testing, which is good. But there are some aspects of testing that still need to be taken care of, one of which is customization testing. Let’s look at what is involved in this in today’s blog.
In one of my earlier blogs, I wrote about ‘generative’ and ‘generated’ applications, and what is involved in testing them. When generative applications provide the boilerplate for the generated applications, most of the generated code is presumably tested in the generative software release. When that application is used to generate another application, the UI items, look and feel, etc., are customized so that the generated application satisfies the needs of the clients. The tests that are done on the generated application to make sure that it is as expected are the customizations tests.
Customization tests ideally should test all the possible combinations in which customizations can be done. One of the challenges involved here is combinatorial complexity. Because of the number of ways in which the customizations can be done, the possible output combinations that need to be tested could become prohibitive, and so, techniques such as pairwise testing may need to employed. But pairwise is not a thorough and complete testing technique, and it can potentially cover only around 70% of the combinations as per the research results available. And hence, it still remains a challenge to figure out which customizations are important to the customers so that they can be tested.
Nevertheless, GUI boilerplates and configuration files such as YAML are the way forward in building applications and systems. They reduce or eliminate the code that needs to be written. There is potential in figuring out ways of testing customizations with rapidly growing number of applications and scenarios. It may be easy to just accept the defaults in them and release the generative applications, but production does not work that way, since users would like to use the customizations as they want. If those have not be thoroughly tested, there will be serious consequences, especially in places like Kubernetes where YAML is used.
KubeCon is happening from Oct 24 to 28th, 2022, and I hope folks discuss the testing aspects of Kubernetes there!
For customization testing strategies for your organisation, feel free to contact me.