In most cases, artificial intelligence has a black-box character. However, it is only through transparency that trust can be built. Specialized software is available in order to explain different AI models. Now, a study by Fraunhofer IPA has compared and evaluated various methods, which explain machine learning approaches.
Just a few decades ago, artificial intelligence (AI) was the stuff of science fiction, but has since become part of our daily lives. In manufacturing, it identifies anomalies in the production process. For banks, it makes decisions regarding loans. And on Netflix, it finds just the right film for every user. All of this is made possible by highly complex algorithms that work covertly in the background. The more challenging the problem, the more complex the AI model – and the more inscrutable.
But users want to be able to understand how a decision has been made, particularly with critical applications: Why was the work piece rejected? What caused the damage to my machine? It is only by understanding the reason for decisions that improvements can be made – and this increasingly applies to safety, too. In addition, the EU General Data Protection Regulation stipulates that decisions must be transparent.
In order to solve this problem, an entirely new area of research has arisen: “Explainable Artificial Intelligence”, or xAI for short. There are now numerous digital aids on the market which explain complex AI approaches. For example, in an image, they mark up the pixel which led to parts being rejected. The Stuttgart-based experts at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA have now compared and evaluated nine common explanation techniques – including LIME, SHAP and Layer-Wise Relevance Propagation – by means of exemplary applications. There were three main criteria for this:
Conclusion of the study: All explanation methods researched were viable. However, as Nina Schaaf, who is responsible for the study at Fraunhofer IPA, explains: “There is no single perfect method.” For example, significant differences emerge in the time needed for generating explanations. The respective objective also largely determines which software is best to use. For example, Layer-Wise Relevance Propagation and Integrated Gradients are particularly good for image data. In summary, Schaaf says: “The target group is also important when it comes to explanations: An AI-developer will want and should receive an explanation phrased differently to the production manager, as both will draw different conclusions from the explanation.”
The study “Explainable AI in Practice. Application-based evaluation of xAI methods” can be accessed in German and English at: https://www.ki-fortschrittszentrum.de/de/studien/erklaerbare-ki-in-der-praxis.html
Fraunhofer Institute for Manufacturing Engineering and Automation IPA