RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación
Contribución de congreso (ConferenceOutput)
When AI Difficulty is Easy: The Explanatory Power of Predicting IRT Difficulty
[EN] One of challenges of artificial intelligence as a whole is robustness. Many issues such as adversarial examples, out of distribution performance, Clever Hans phenomena, and the wider areas of AI evaluation and explainable AI, have to do with the following question: Did the system fail because it is a hard instance or because something else? In this paper we address this question with a generic method for estimating IRT-based instance difficulty for a wide range of AI domains covering several areas, from supervised feature-based clas- sification to automated reasoning. We show how to estimate difficulty systematically using off-the-shelf machine learning regression models. We illustrate the usefulness of this estima- tion for a range of applications., This work has been partially supported by the EU (FEDER) and Spanish MINECO grant RTI2018-094403-B-C32 funded by MCIN/AEI/10.13039/501100011033 and by ERDF A way of making Europe , Generalitat Valenciana under grant PROMETEO/2019/098, EU s Horizon 2020 research and innovation programme under grant agreement No. 952215 (TAILOR), the Future of Life Institute, FLI, under grant RFP2-152, US DARPA HR00112120007 (RECoG-AI), the AI-Watch and HUMAINT projects by DG CONNECT and DG JRC of the European Commission, and INNEST/2021/317 (Project cofunded by the European Union with the Programa Operativo del Fondo Europeo de Desarrollo Regional (FEDER) de la Comunitat Valenciana 2014-2020).
Proyecto:
Agencia Estatal de Investigación / Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 RTI2018-094403-B-C32-AR
,
Generalitat Valenciana / Subvenciones del programa Prometeo para grupos de investigación de excelencia – PROMETEO 2019 PROMETEO/2019/098
,
European Commission / H2020 952215
,
Generalitat Valenciana INNEST/2021/317
,
U.S. Department of Defense HR00112120007