The latest survey by Weber Shandwick shows that Generative AI is a top priority for CEOs, with 51% marking it as a top priority above the other options of inflation, ESG, and geo-political conflicts. But most of the CEOs are not sure about how to bell the generative AI cat! Let’s look at this in a bit more detail in this article.
To quote the words of Chris Deri, president of Weber Shandwick’s Corporate Advisory Businesses,
The big takeaway on generative AI…is that 75% of executives do not feel personally equipped to lead their organizations through this. I think that means 2024 is going to be something of an observation year. The No. 1 action executives are looking for is an AI vendor. But they can’t farm out or delegate innovation. They really have to become technologists themselves and have the ethos of testing and learning and failing fast in order to understand.
As Quoted In Fortune CEO Daily
I would like to highlight the last line, the most significant for us, the technologists/testers/quality professionals. The CEOs ought to become technologists themselves and have the ethos to test, learn, and fail fast, in order to understand how the generative AI works.
Let’s take a moment to grasp the intensity of the task involved in testing generative AI. If it is a domain-specific LLM, it would be much easier as the scope is limited, but if it is something like general LLM model like chatGPT, would you even venture to test it yourself, even if you limit yourself to your domain or business? And if you indeed test it, what would you do if you find a fault? And then think about the nightmare of integrating it to your product through an API while you are not sure whether the answers the LLM provides are accurate or not!
For now, as Chris pointed out, it’s going to be an observation period. We are not really sure when something like improving the accuracy, or giving an SLA to a customer saying that my generative-AI based solution would work according to the contracted agreement, would work out!
In India, there are multiple applications that are being developed based on the local needs. It is very heartening to see that young generation is starting up with a wealth of products AND venture capital is pouring in to India. From what I hear, many Indians who had spent significant time in USA and have returned to India are saying that India is the right place to be to tap into the inflow of capital, but time has got to tell whether this momentum is going to work out.
I am putting my testing mind to rest with a conclusion that I would NOT test a GENERIC LLM, as it sounds next to impossible. I wouldn’t try to bell the generative AI cat! Maybe they should write an LLM to test another LLM!
For thoughts on testing and quality, please feel free to get in touch with me.