Personalized Suggestions Testing is key about the feature offered by apps that use AI. In Apple Intelligence in iPhone 16.0 and elsewhere, personalized suggestions is important, as it is being used for
- Suggesting frequently used apps
- Notifications based on the time, location, or activity
- Recommendations during search (apps and widgets)
- Predictive text
When I was looking at devising a detailed test strategy for some of these, I found that the cases are quite exhaustive, and sometimes, even prohibitive. But as always as a curious tester, it is important to come up with majority cases of test coverage to improve the quality posture of the application at hand.
Suggesting frequently used apps
There are tons of ways that an application can figure the apps to suggest as the frequently used ones. The key is in collecting the context, keywords, and images that are being looked for over a period of time.
The best useful feature is to display the apps that have been used recently on the top of the app list. There are two things to consider – how recently the app was used, and how frequently the app was used. If there is an option to finetune which ones should be shown in the top of the list, then that feature should be tested first.
Then, based on which option has been selected, it should be tested whether the recently used apps are being used by rigorously and thoroughly testing with several combinations which include
- Whether all apps that are used are being displayed
- When an app is used but uninstalled, whether it disappears from the list
- Unused apps are not shown in the list by mistake
- No. of apps configured to be displayed in the list and whether the no. of apps displayed is correct, based on various combinations
Recently used apps testing for personalized suggestions
The testing for recently used apps involves testing whether the app is being displayed on the top based on how recently it was used irrespective of how frequently it was used. Even if the app was launched once in the recent past, it should figure in the list. Apps that were never used should not be displayed in the list, and apps that were not used in the recent past (based on certain timeframe, or limited by the no. of apps displayed – depending on how the feature works) also should not be displayed in the list.
Frequently used apps testing for personalized suggestions
In this case, the no. of times the app is used is the parameter to look for, and accordingly, the apps that are most used should be displayed in the descending order based on the number of launches. It should be tested if apps are being replaced in the list based on the usage times. No. of apps displayed should depend on the frequency of usage, and apps that are not used should not displayed in the list.
Suggestions based on context
There is one twist to this story which is that, if the app is watching the context (time, location, text being typed, etc.), it could potentially suggest apps which were neither recently used nor frequently used! One has to check how the feature works in the specific implementation that is being tested and arrive at a conclusion on how to test this. It is recommended that the user go through the detailed documentation on how the feature works.
Last, but not the least…
I am always excited about coming up with the test strategy and scenarios of apps based on AI and GenAI. In the future, I will write about notifications, search recommendations, and predictive text. If your organisation has a need to devise test strategy for personalized suggestions testing or related areas in apps, please feel free to get in touch with me.