To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods

View article
Loading...
PeerJ Computer Science

Main article text

 

Introduction

Background

Local linear explanation methods

LIME: local interpretable model-agnostic explanations

SHAP: shapley additive explanations

On feature importance

Methodology

  • P1: Explaining single decisions. The black-box classifier f is given. The goal is to understand if an explanation g computed by an explainer on a single instance x can be trusted, and how complex g should be. LEAF supports this use case by providing metrics related to the quality of the explanation, given that the LLE method and a level of explanation complexity are defined. This allows an end-user to trust the white-box explanation, or to reject it if it does not meet the expected quality.

  • P2: Model development. The black-box model f can be chosen to maximise both classification performances and model interpretability. In this case, it might be important to know which classifier should be used to provide, on average, the best explanations. LEAF supports this use case with a feedback loop allowing the comparison of different black-box models in terms of quality of the explanations that can be extracted.

Conciseness

Local fidelity

Local concordance

Reiteration similarity

Prescriptivity

Results

Evaluating the reiteration similarity of XAI methods

Evaluating the prescriptivity of explanations

Selecting an explainable classifier

  • What explainability method should be selected in order to have stable explanations? LIME is highly stable only for linear/logistic classifiers, and for lower conciseness levels (higher K values). SHAP instead shows excellent reiteration similarity even at low conciseness.

  • What classifier should be trained to have high accuracy in the LLE models? LIME has low local concordance for some classifiers (kn, mlp, svc), even for high values of K. SHAP instead shows increasing concordance levels at the increase of K, for all classifiers. Therefore SHAP is a better choice for local concordance, unless the black-box classifier is a linear or a logistic one.

  • What choices should be made to have explanations with high local fidelity? Surprisingly, high local fidelity explanations can only be achieved by using some classifier categories (linear, logistic, and random forest for LIME). Increasing the conciseness does not appear to increase the local fidelity significantly, at least in the tested range of K. Some classifier categories (mlp, svc) show very poor explanation fidelities, regardless of the explainability method used.

  • What choices should be made to have prescriptive explanations? Again, explanations can be used in a prescriptive way only for some classifier categories (lin, log). SHAP appears to have high prescriptive power even for low values of K, but only for a few classifier categories (lin, log and moderately for rf). LIME requires higher values of K than SHAP to generate prescriptive explanations consistently. Other classifier categories (kn, mlp, svc) have poorly prescriptive LLE models, independently of the value of K and the method used.

Discussion

Conclusions

Additional Information and Declarations

Competing Interests

The authors declare there are no competing interests.

Author Contributions

Elvio Amparore conceived and designed the experiments, performed the experiments, analyzed the data, performed the computation work, prepared figures and/or tables, authored or reviewed drafts of the paper, and approved the final draft.

Alan Perotti conceived and designed the experiments, analyzed the data, authored or reviewed drafts of the paper, and approved the final draft.

Paolo Bajardi conceived and designed the experiments, analyzed the data, prepared figures and/or tables, authored or reviewed drafts of the paper, and approved the final draft.

Data Availability

The following information was supplied regarding data availability:

The source code and data is available at GitHub: https://github.com/amparore/leaf.

Funding

This work was supported by the Regional Research Project “Casa Nel Parco” (POR FESR 14/20 - CANP - Cod. 320 - 16 - Piattaforma Tecnologica “Salute e Benessere”) funded by Regione Piemonte in the context of the Regional Platform on Health and Well-being and from Intesa Sanpaolo Innovation Center. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

59 Citations 4,019 Views 1,111 Downloads