RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication
Artículo científico (JournalArticle)
Rethink reporting of evaluation results in AI
[EN] Artificial intelligence (AI) systems have
begun to be deployed in high-stakes
contexts, including autonomous driving and medical diagnosis. In contexts
such as these, the consequences of
system failures can be devastating. It
is therefore vital that researchers and policymakers have a full understanding of the capabilities and weaknesses of AI systems so
that they can make informed decisions about
where these systems are safe to use and how
they might be improved., We are grateful to J. Pineau and S. Russell for their feedback
on the manuscript and to M. Brundage and L. Ahmad for
helpful discussions about the need for instance-by-instance
results for foundation models. Funding support was provided
by the Future of Life Institute under grant RFP2-152 (J.H.O.
and J.B.); US Defense Advanced Research Projects Agency
HR00112120007 (RECoG-AI) (J.H.O., L.C., R.B., J.B., and
D.K.); grant PID2021-122830OB-C42 (SFERA) funded by
MCIN/AEI/10.13039/501100011033 and ERDF A way of
making Europe (J.H.O., F.M.P., and W.S.); INNEST/2021/317
[Project cofunded by the European Union with the Programa
Operativo del Fondo Europeo de Desarrollo Regional (FEDER)
de la Comunitat Valenciana 2014-2020 ] (J.H.O. and W.S.);
and UPV (Vicerrectorado de Investigación) grant PAI-10-21
(J.H.O. and W.S.).
Project:
Agencia Estatal de Investigación / Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2021-122830OB-C42
,
Universitat Politècnica de València PAID-10-21
,
Generalitat Valenciana INNEST/2021/317
,
Future of Life Institute RFP2-152
,
U.S. Department of Defense HR00112120007