Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 5
Encontrada(s) 1 página(s)

Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publication Capítulo o parte de un libro (BookPart)

Beyond Prediction Similarity: ShapGAP for Evaluating Faithful Surrogate Models in XAI

  • Mariotti, Ettore
  • Sivaprasad, Adarsa
  • Alonso Moral, José María
The growing importance of Explainable Artificial Intelligence (XAI) has highlighted the need to understand the decision-making processes of black-box models. Surrogation, emulating a black-box model (BB) with a white-box model (WB), is crucial in applications where BBs are unavailable due to security or practical concerns. Traditional fidelity measures only evaluate the similarity of the final predictions, which can lead to a significant limitation: considering a WB faithful even when it has the same prediction as the BB but with a completely different rationale. Addressing this limitation is crucial to develop Trustworthy AI practical applications beyond XAI. To address this issue, we introduce ShapGAP, a novel metric that assesses the faithfulness of surrogate models by comparing their reasoning paths, using SHAP explanations as a proxy. We validate the effectiveness of ShapGAP by applying it to real-world datasets from healthcare and finance domains, comparing its performance against traditional fidelity measures. Our results show that ShapGAP enables better understanding and trust in XAI systems, revealing the potential dangers of relying on models with high task accuracy but unfaithful explanations. ShapGAP serves as a valuable tool for identifying faithful surrogate models, paving the way for more reliable and Trustworthy AI applications.
Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publication Artículo científico (JournalArticle)

On the role of explanations in machine learning prediction of nonalcoholic steatohepatitis

  • Babakov, Nikolay
  • Rezgova, Elena
  • Reiter, Ehud
  • Bugarín-Diz, Alberto
Nonalcoholic steatohepatitis (NASH) is a common disease that ultimately can lead to the development of end-stage liver disease, cirrhosis, or hepatocellular carcinoma. An early prediction of NASH provides an opportunity to make an appropriate strategy for prevention, early diagnosis, and treatment. The most accurate approach for NASH diagnostics is a liver biopsy, which can lead to various complications for the patient. Many papers have studied non-invasive machine learning (ML)-driven approaches to early non-invasive NASH prediction; however, to the best of our knowledge, none of the works considered the problem of explainability of the trained ML models to the medical experts. In this work, we address this issue. We use the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) nonalcoholic fatty liver disease adult database to train different ML models and propose the technique to explain their predictions. We compare the explanations obtained from a transparent model (Decision Tree) and a non-transparent model (Random Forest). Furthermore, we analyze the quality of explanation prediction by objective means and with a user study involving 11 medical practitioners. Our findings show that there is no significant difference in the perception of explanation obtained from transparent and non-transparent models, and that the explanation of the models’ predictions slightly increases their usability and trustworthiness for real practitioners, enhancing their practical adoption in clinical settings.
Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publication Artículo científico (JournalArticle)

Exploring the balance between interpretability and performance with carefully designed constrainable Neural Additive Models

  • Mariotti, Ettore
  • Alonso Moral, José María
  • Gatt, Albert
The interpretability of an intelligent model automatically derived from data is a property that can be acted upon with a set of structural constraints that such a model should adhere to. Often these are in contrast with the task objective and it is not straightforward how to explore the balance between model interpretability and performance. In order to allow an interested user to jointly optimise performance and interpretability, we propose a new formulation of Neural Additive Models (NAM) which can be subject to a number of constraints. Accordingly, our approach produces a new model that is called Constrainable NAM (or just CNAM in short) and it allows the specification of different regularisation terms. CNAM is differentiable and is built in such a way that it can be initialised as a solution of an efficient tree-based GAM solver (e.g., Explainable Boosting Machines). From this local optimum the model can then explore solutions with different interpretability-performance tradeoffs according to different definitions of both interpretability and performance. We empirically benchmark the model on 56 datasets against 12 models and observe that on average the proposed CNAM model ranks on the Pareto front of optimal solutions, i.e., models generated by CNAM exhibit a good balance between interpretability and performance. Moreover, we provide two illustrative examples which are aimed to show step by step how CNAM works well for solving classification tasks, but also how it can yield insights when considering regression tasks
Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publication Capítulo o parte de un libro (BookPart)

A Confusion Matrix for Evaluating Feature Attribution Methods

  • Arias-Duart, Anna
  • Mariotti, Ettore
  • García-Gasulla, Darío
  • Alonso Moral, José María
The increasing use of deep learning models in critical areas of computer vision and the consequent need for insights into model behaviour have led to the development of numerous feature attribution methods. However, these attributions must be both meaningful and plausible to end-users, which is not always the case. Recent research has emphasized the importance of faithfulness in attributions, as plausibility without faithfulness can result in misleading explanations and incorrect decisions. In this work, we propose a novel approach to evaluate the faithfulness of feature attribution methods by constructing an ‘Attribution Confusion Matrix’, which allows us to leverage a wide range of existing metrics from the traditional confusion matrix. This approach effectively introduces multiple evaluation measures for faithfulness in feature attribution methods in a unified and consistent framework. We demonstrate the effectiveness of our approach on various datasets, attribution methods, and models, emphasizing the importance of faithfulness in generating plausible and reliable explanations while also illustrating the distinct behaviour of different feature attribution methods.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

An explanation-oriented inquiry dialogue game for expert collaborative recommendations

  • Shaheen, Qurat Ul Ain
  • Budzynska, Katarzyna
  • Sierra, Carles
This work presents a requirement analysis for collaborative dialogues among medical experts and an inquiry dialogue game based on this analysis for incorporating explainability into multiagent system design. The game allows experts with different knowledge bases to collaboratively make recommendations while generating rich traces of the reasoning process through combining explanation-based illocutionary forces in an inquiry dialogue. The dialogue game was implemented as a prototype web-application and evaluated against the specification through a formative user study. The user study confirms that the dialogue game meets the needs for collaboration among medical experts. It also provides insights on the real-life value of dialogue-based communication tools for the medical community., The work reported in this paper has been supported in part by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No. 860621, in part by the project 2021 SGR 00754 of the Catalan Government, in part by Chist-Era under grant 2022/04/Y/ST6/00001, in part by POB CyberDS of Warsaw University of Technology within the Excellence Initiative: Research University (IDUB) programme under grant 1820/1/Z01/POB3/2021, and in part by VW foundation (VolkswagenStiftung) under grant 98 542., Peer reviewed