Managing user expectations as to what the product should offer, directly translates to the options or choices that the users would have and the Quality of the product. The more choices you have, there are more chances of risks, dissatisfaction, and defects. This is natural and hence as Software professionals, we need to be cognizant of managing user expectations and having an optimal number of features, options, and choices that the product would offer towards a great user experience.
Giving too many input options confuses the user. Let’s take the example of you going to a coffee shop and ordering coffee. You would expect the waiter asking you some basic questions and take the order, so that they can be on their way to make the coffee that you want, but they ask you:
- What kind of flavor?
- Light or dark?
- Watery or thick?
- Hot or cold?
- With or without ice cubes?
- Do you want some ice-cream added to it?
- With sugar or without sugar?
- With milk or without milk?
- How do you like to brew?
- Lemon in it?
- How about some cream?
- ……
And you get irritated and ask them to just bring the damn coffee!
Software is no different. Because of the complexities of the technologies involved, you got to be careful on what kind of questions are being asked. You will have all types of users encountering it – from novice to experts. And it’s best to limit the questions to basics and let the system figure out the rest.
The YAML configuration files have become so popular. I saw a funny GIF in Twitter that showed a skeleton and said ‘This was a person who had been trying to configure a Kubernetes YAML file’. The number of options is too huge that most of the users don’t touch many of the options and let the default options be. Because they don’t understand the options, they don’t know what the software is doing. And there are companies that thrive on configuring Kubernetes YAML files for you!
The real question here is what are we trying to do with the software, and why are we allowing the users to tweak the software with so many options. If we don’t manage the number of options that we provide the users, we have the nightmare of having a software that has so many risks, failure possibilities, and defects. Sure, you can deliver each of these options in a very incremental, iterative way in the CI/CD pipeline, with multiple teams checking in at the same time, but when everything comes together, what kind of solution space would you have? How many number of possible configurations that you would test? Before solution level testing, how does your integration testing look like – features one to one, one to some, one to all, some to some, some to all, all to all? It is just mind-boggling.
Not just the software, your product documentation also becomes a nightmare because of the combinations you need to cover. How do you introduce these features in a meaningful and user-friendly way to the end-user? ‘No, you don’t. That’s why we have an expert plan that would manage your software for US$2000 a month. We also have certifications and books!’ Ha! We get it! The idea seems to be making the software unnecessarily complex, and then generate experts, courses, and books out of it.
When I was doing my college final project, my collegemate looked at my project and said, ‘But Venkat, aren’t you supposed to ask the least number of questions to the users? Why do you ask so many questions? The software should figure it out’. I was trying to build an expert system (with a knowledge-base) by the way, and the system was asking the user the symptoms that they are seeing, so that it could come up with a conclusion with the highest probability. What my collegemate said may not be right for that kind of systems, because those kind of systems should ask questions to figure out the scenario, but for other types of systems, you need not or shouldn’t be asking too many questions and go with a blue-print. My collegemate’s question still is in my mind when I look at Software for testing and Quality.
What can we, as Software Testers, do to influence design considerations when the product is being developed? We can help in asking questions about:
- Is this option required?
- What percentage of users use what options? What is the market risk if the product assumes a specific option and does not give a choice to the user? Have we done any market survey to figure out?
- What are the combinatorial explosion risks that we would face with the number of options we provide? How do we integration test them? How do we solution test them?
- What are the underlying tech. stack challenges? Would it work with all operating systems, browsers, hardware, …..
Playing the role of supportive questioning to let the developers figure out these details upfront in managing user expectations will save a lot of time and effort in designing, implementing, testing, debugging, and in production.
I am using a product right now, and I see a lot of defects and a lot of issues in documentation, and I can tell that their team is struggling in juggling with multiple features with the number of developers that they have. This is true with startups, and hence more due-diligence upfront will be helpful in managing customer expectations and reducing the number of options.
Feel free to contact me for quality consulting on managing product expectations.