I came across this podcast about human oversight on ML-based decisions, and I felt it’s awesome. However sophisticated the ML algorithm be, it’s always necessary to run the final decision through a human expert. It also helps in finetuning the algorithm when differences occur between the human expert and the algorithm. This impacts Software Quality.
From a software testing perspective, it is very helpful to have the expert in the loop. In addition to finetuning the algorithm, the algorithm can also be augmented with additional parameters as required based on the expert inputs. Data patterns keep varying, and the parameters that we need to look to make the decision would also vary depending on how the experts pick the parameters and make the decisions, and the algorithm has to follow through, and so would the testing team to make sure if things are right. Otherwise, the algorithm would become stale.
From a Software Quality perspective too, it’s very important to have each and every decision checked by a human. It’s not really about the percentages of how many false-positives or true-negatives happened, because each and every decision could be related to something very important like detecting cancer or making a decision on whether to issue a big loan by a bank.
When it’s done on real time, it’s very difficult as the humans have to make spot decisions in thousands such as in an assembly line checking for faulty parts. But then, they have to do it anyway even without ML. ML, if implemented correctly, would potentially reduce the fatigue associated with checking and making these decisions on a day-to-day basis. Of course, there could a small margin of error associated with the ranges; some kind of tolerance level, which will keep getting smaller and smaller as the algorithm becomes very effective.
Talk to me about testing Machine Learning implementations on decision-making related to various sectors in the Logistics & Supply Chain industry, and about Software Quality. Look forward to hear from you!
