Why Testing NLP Is Daunting

While processing customer feedback or comments that are in text, and finding how to improve customer satisfaction with in-built capabilities in CRM and other systems, we get to a stage where we feel helpless because of the inability to process language beyond a certain point. This can be clearly seen in languages like English where there are multiple uses of the same word (e.g. “Very”) with different uses (adverb, adjective) in different contexts. Let’s talk about this complexity in testing Machine Learning in this blog.

The way the Artificial Neural Networks process the text is not constructing/re-constructing the sentences. Research has been going on on how to do this in fact, but this too has faced roadblocks because of the contextual usage, and most often the way people use the language in colloquial ways, not necessarily with perfect grammar. Add to that the complexities raised by things like spelling mistakes, and you got a monster in your hands to deal with.

Well, first of all, developing a system in itself is complex, and then think about testing it! When I spoke about Testing NLP as a Keynote speaker in PSTC 2018 in Pune, most people felt incredulous about what I was suggesting. Well, they wanted a quick-fix mantra or a magic wand to make this problem go away, or probably they wanted to add a tool’s name in their testing resume so that it would look better. Unfortunately, that is not going to happen!

I suggested that in case of simple scenarios like processing of text to select among a few options, it’s better to go for a hierarchy of selections like we always traditionally did (Amazon does this in their customer service feedback/complaint panel), and not use things like chatbots. Because what we are doing here is customer service, and not rocket science. I told the same thing when I did a talk in Thoughtworks, Bangalore.

So, yes, we love technology, and we love challenges, but let’s also be practical. I know that there are great things coming up in NLP, and believe me I have tried some of them. They indeed LOOK versatile at the surface, but if you poke them a bit, they fail miserably.

In spite of all this warning, if you are still a chatbot development aspirant, go ahead. You will only be developing chatbots for the next few coming decades, may be never-ending!

If you would like to get my advisory inputs for your company testing Machine Learning based products, give me a shout, and let me see how I can help you.