Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 23
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Aggregating Value Systems for Decision Support

  • Lera-Leri, Roger X.
  • Liscio, Enrico
  • Bistaffa, Filippo
  • Jonker, Catholijn M.
  • Lopez-Sanchez, Maite
  • Murukannaiah, Pradeep K.
  • Rodriguez-Aguilar, Juan A.
  • Salas-Molina, Francisco|||0000-0002-1168-7931
[EN] We adopt an emerging and prominent vision of human -centred Artificial Intelligence that requires building trustworthy intelligent systems. Such systems should be capable of dealing with the challenges of an interconnected, globalised world by handling plurality and by abiding by human values. Within this vision, pluralistic value alignment is a core problem for AI- that is, the challenge of creating AI systems that align with a set of diverse individual value systems. So far, most literature on value alignment has considered alignment to a single value system. To address this research gap, we propose a novel method for estimating and aggregating multiple individual value systems. We rely on recent results in the social choice literature and formalise the value system aggregation problem as an optimisation problem. We then cast this problem as an l(p)-regression problem. Doing so provides a principled and general theoretical framework to model and solve the aggregation problem. Our aggregation method allows us to consider a range of ethical principles, from utilitarian (maximum utility) to egalitarian (maximum fairness). We illustrate the aggregation of value systems by considering real -world data from two case studies: the Participatory Value Evaluation process and the European Values Study. Our experimental evaluation shows how different consensus value systems can be obtained depending on the ethical principle of choice, leading to practical insights for a decision -maker on how to perform value system aggregation., The authors were supported by the research projects ACISUD (PID2022-136787NB-I00) , VAE (TED2021-131295B-C31) , VALAWAI (HE-101070930) , Fairtrans (PID2021-124361OB-C33) , Yoma OR (OPE02570) , TAILOR (952215) , and the Hybrid Intelligence Center, a 10-year programme funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research. Maite Lopez-Sanchez belongs to the WAI research group (University of Barcelona) , an associated unit to CSIC through the IIIA.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Towards Agrirobot Digital Twins: Agri-RO5 A Multi-Agent Architecture for Dynamic Fleet Simulation

  • Gutiérrez Cejudo, Jorge
  • Lujak, Marin
  • Fernandez, Alberto
  • Enguix-Andrés, Francisco|||0009-0002-8144-318X
  • Carrascosa Casamayor, Carlos|||0000-0003-3649-6530
  • Hérnandez López, Luís|||0000-0001-9011-0542
[EN] In this paper, we propose a multi-agent-based architecture for a Unity3D simulation of dynamic agrirobot-fleet-coordination methods. The architecture is based on a Robot Operating System (ROS) and Agrobots-SIM package that extends the existing package Patrolling SIM made for multi-robot patrolling. The Agrobots-SIM package accommodates dynamic multi-robot task allocation and vehicle routing considering limited robot battery autonomy. Moreover, it accommodates the dynamic assignment of implements to robots for the execution of heterogeneous tasks. The system coordinates task assignment and vehicle routing in real time and responds to unforeseen contingencies during simulation considering dynamic updates of the data related to the environment, tasks, implements, and robots. Except for the ROS and Agrobots-SIM package, other crucial components of the architecture include SPADE3 middleware for developing and executing multi-agent decision making and the FIVE framework that allows us to seamlessly define the environment and incorporate the Agrobots-SIM algorithms to be validated into SPADE agents inhabiting such an environment. We compare the proposed simulation architecture with the conventional approach to 3D multi-robot simulation in Gazebo. The functioning of the simulation architecture is demonstrated in several use-case experiments. Even though resource consumption and community support are still an open challenge in Unity3D, the proposed Agri-RO5 architecture gives better results in terms of simulation realism and scalability., This work was partially supported by grants PID2021-123673OB-C32, PID2021-123673OBC33, TED2021-131295B-C31 and TED2021-131295B-C33 funded by MCIN/AEI/10.13039/501100011033 and by ERDF A way of making Europe and the European Union NextGenerationEU/PRTR , respectively, as well as the Agrobots Project funded by the Community of Madrid, Spain, and the AGROBOTIX Project funded by the University Rey Juan Carlos.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

A general approach for computing a consensus in group decision making that integrates multiple ethical principles

  • Salas-Molina, Francisco|||0000-0002-1168-7931
  • Bistaffa, Filippo
  • Rodríguez-Aguilar, Juan A.
[EN] We tackle the problem of computing a consensus according to multiple ethical principles - which can include, for example, the principle of maximum freedom associated with the Benthamite doctrine and the principle of maximum fairness associated with the Rawlsian principles - among the preferences of different individuals in the context of Group Decision-Making (GDM). More formally, we put forward a novel formalisation of the above-mentioned problem based on a multi-lp-norm approximation problem that aims at minimising multiple p-metric distance functions, where each parameter p represents a given ethical principle. Our contribution incurs obvious benefits from a social-choice perspective. Firstly, our approach significantly generalises stateof-the-art approaches that were limited to only two ethical principles (p = 1, for maximum freedom, and p = & INFIN;, for maximum fairness). Secondly, our experimental results considering an established test case demonstrate that our approach is capable, thanks to a novel re-weighting scheme, to compute a multi-norm consensus that takes into account each ethical principle in a balanced way, in contrast with state-of-the-art approaches that were heavily biased towards the p =1 ethical principle., Research supported by projects: CI-SUSTAIN, Spain (PID2019-104156GB-I00) ; TAILOR, Spain (H2020-952215) ; 2021 SGR 00754 funded by Generalitat de Catalunya, Spain; VAE TED2021-131295B-C31, funded by MCIN/AEI, Spain/10.13 039/501100011033 and NextGenerationEU/PRTR, Spain; and VALAWAI, Spain (Horizon Europe #101070930) . Funding for open access charge: CRUE-Universitat Politecnica de Valencia.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Otros (Other)

Towards Convention-Based Game Strategies

  • Pan, Shuxian
  • Sierra, Carles
27th International Workshop, COINE 2023, London, UK, May 29, 2023, Revised Selected Papers., To effectively develop cooperative multiagent systems, we introduce an architecture that facilitates the agents’ dynamic adoption of conventions. It expands an existing agent model’s action selection architecture with a component that uses Natural Language Processing techniques. This component embeds conventions into agent interaction strategies to improve the predictability of other agents’ actions if all agents adopt the same conventions in their strategies., This research is conducted under the REDI Program, a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sk lodowska-Curie grant agreement no. 101034328. This paper reflects only the author’s view and the Research Executive Agency is not responsible for any use that may be made of the information it contains. This research also receives support from the VALAWAI project (Horizon #101070930) and the VAE project (Grant no. TED2021-131295 B-C31) funded by MCIN/AEI/10.13039/501100011033., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Aggregating value systems for decision support

  • Lera-Leri, Roger X.
  • Liscio, Enrico
  • Bistaffa, Filippo
  • Jonker, Catholijn M.
  • López-Sánchez, Maite
  • Murukannaiah, Pradeep K.
  • Rodríguez-Aguilar, Juan Antonio
  • Salas-Molina, Francisco
We adopt an emerging and prominent vision of human-centred Artificial Intelligence that requires building trustworthy intelligent systems. Such systems should be capable of dealing with the challenges of an interconnected, globalised world by handling plurality and by abiding by human values. Within this vision, pluralistic value alignment is a core problem for AI– that is, the challenge of creating AI systems that align with a set of diverse individual value systems. So far, most literature on value alignment has considered alignment to a single value system. To address this research gap, we propose a novel method for estimating and aggregating multiple individual value systems. We rely on recent results in the social choice literature and formalise the value system aggregation problem as an optimisation problem. We then cast this problem as an ℓp-regression problem. Doing so provides a principled and general theoretical framework to model and solve the aggregation problem. Our aggregation method allows us to consider a range of ethical principles, from utilitarian (maximum utility) to egalitarian (maximum fairness). We illustrate the aggregation of value systems by considering real-world data from two case studies: the Participatory Value Evaluation process and the European Values Study. Our experimental evaluation shows how different consensus value systems can be obtained depending on the ethical principle of choice, leading to practical insights for a decision-maker on how to perform value system aggregation., The authors were supported by the research projects ACISUD (PID2022-136787NB-I00), VAE (TED2021-131295B-C31), VALAWAI (HE-101070930), Fairtrans (PID2021-124361OB-C33), Yoma OR (OPE02570), TAILOR (952215), and the Hybrid Intelligence Center, a 10-year programme funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research ., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Is this a violation? Learning and understanding norm violations in online communities

  • Freitas dos Santos, Thiago
  • Osman, Nardine
  • Schorlemmer, Marco
Using norms to guide and coordinate interactions has gained tremendous attention in the multi-agent community. However, new challenges arise as the interest moves towards dynamic socio-technical systems, where human and software agents interact, and interactions are required to adapt to human's changing needs. For instance, different agents (human or software) might not have the same understanding of what it means to violate a norm (e.g., what characterizes hate speech), or their understanding of a norm might change over time (e.g., what constitutes an acceptable response time). The challenge is to address these issues by learning the meaning of a norm violation from limited interaction data. For this, we use batch and incremental learning to train an ensemble of classifiers. Ensemble learning and data-sampling handle the imbalanced class distribution of the interaction stream. At the same time, the training approaches use different strategies to ensure that the ensemble models reflect the latest community view on the meaning of norm violation. Batch learning uses weight assignment, while incremental learning continuously updates the ensemble models as community members interact. Here, we extend our previous work by creating a different balance strategy for online learning and integrating interpretability to understand norm violations. Additionally, we evaluate the proposed approaches in the context of Wikipedia article edits, where interactions revolve around editing articles, and the norm in question is prohibiting vandalism. Lastly, we conduct ablation studies to compare the ensemble's performance against a single model approach and to examine the behavior of two data sampling techniques. Results indicate that the different machine learning frameworks can learn the meaning of a norm violation in a setting with data imbalance and concept drift, although with significant differences., This research is supported by the EU-funded VALAWAI (# 101070930) and WeNet (# 823783) projects, the Spanish-funded VAE (# TED2021-131295B-C31) and Rhymas (# PID2020-113594RB-100) projects, and the Generalitat de Catalunya funded Ajuts a grups de recerca de Catalunya (# 2021 SGR 00754) project., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Theoretical properties of the MiCRO negotiation strategy

  • De Jonge, Dave
Recently, we have introduced a new algorithm for automated negotiation, called MiCRO, which, despite its simplicity, outperforms many state-of-the-art negotiation strategies (de Jonge, in: Raedt (ed) Proceedings of the thirty-first international joint conference on artificial intelligence, ijcai.org, Vienna, Austria, 2022). Furthermore, we claimed that under certain conditions which typically hold in the Automated Negotiating Agents Competition (ANAC), it is a game-theoretically optimal strategy. The goal of this paper is to formally prove those claims. Specifically, we define ‘negotiation’ as an extensive-form game and define the class of consistent strategies for this game, which consists of those strategies that satisfy a number of rationality criteria. We then prove that under the above mentioned conditions MiCRO is a best response against itself among all consistent negotiation strategies. Furthermore, we define the notion of a balanced negotiation domain, which is a domain in which two MiCRO agents would always come to an optimal agreement. Finally, we show that many of the domains used in ANAC indeed happen to be (approximately) balanced. The importance of this work is that if we know under which conditions MiCRO is theoretically optimal, then we can use this to test to what extent other negotiation algorithms are able to achieve similar results to MiCRO when applied under those same conditions. Furthermore, it would help researchers to design more challenging test cases for automated negotiation in which MiCRO is not optimal., This work was supported by a Juan de la Cierva - Incorporación research grant from the Spanish Ministry of Science and Innovation (IJC2018-036443-I) and by Grant no. TED2021-131295B-C31 funded by MCIN/AEI /10.13039/501100011033 and by the European Union NextGenerationEU/PRTR., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

El valor de la ética aplicada en los estudios de ingeniería en un horizonte de inteligencia artificial confiable, The value of applied ethics in engineering studies in a horizon of reliable artificial intelligence

  • Terrones Rodríguez, Antonio Luis
  • Rocha Bernardi, Mariana
Este artículo contiene un PDF adicional en inglés., [EN] Political institutions such as the European Commission and the Government of Spain have expressed their interest and willingness to lay the foundations for ethical governance of Artificial Intelligence (AI). In particular, they have proposed the promotion of reliable Artificial Intelligence through a set of guidelines and strategies. Despite the benefit provided by these political initiatives, it is not possible to fully appreciate a specific educational strategy that contributes to the generation of an ethical AI ecosystem based on trust. In this sense, the main objective of this work is to demonstrate that the teaching of applied ethics in engineering studies constitutes a commitment to professional strengthening, ethical governance and responsible research and innovation. To achieve this objective, first of all, some of the essential aspects that give significant value to the teaching of applied ethics will be detailed, highlighting the social responsibility that universities present in this field. Secondly, a classification of ethics subjects related to engineering in the environment of Spanish public universities will be shown. And, thirdly, a theoretical framework rooted in discursive ethics will be offered, promoting a civic perspective in the educational context of the professions, [ES] Instituciones políticas como la Comisión Europea y el Gobierno de España han manifestado su interés y predisposición para sentar las bases de una gobernanza ética de la Inteligencia Artificial (IA). En particular, han planteado el impulso de una Inteligencia Artificial confiable a través de un conjunto de directrices y estrategias. A pesar del beneficio que reportan estas iniciativas políticas, no es posible apreciar en su conjunto una estrategia educativa específica que contribuya a la generación de un ecosistema ético de IA fundamentado en la confianza. En ese sentido, el objetivo principal de este trabajo consiste en explicar que la enseñanza de la ética aplicada en los estudios de ingeniería constituye una apuesta para el fortalecimiento profesional, la gobernanza ética y una investigación e innovación responsables. Para alcanzar este objetivo, en primer lugar, serán detallados algunos de los aspectos esenciales que dotan de un valor significativo a la enseñanza de la ética aplicada, subrayando la responsabilidad social que presentan las universidades en este terreno. En segundo lugar, se mostrará una clasificación de las asignaturas de ética relacionadas con la ingeniería en el entorno de las universidades públicas españolas. Y, en tercer lugar, se ofrecerá un marco teórico enraizado en la ética discursiva, promoviendo una mirada cívica en el contexto educativo de las profesiones., Entidad financiadora: MCIN/AEI/10.13039/501100011033 y la Unión Europea NextGenerationEU/PRTR Proyecto subvencionado: Ingeniería de Valores en Sistemas de IA-VAE Código de proyecto: TED2021-131295B-C31, Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Otros (Other)

On autonomy, governance and values; an AGV approach to value engineering

  • Noriega, Pablo
  • Plaza, Enric
First International Workshop, VALE 2023, Krakow, Poland, September 30, 2023, Proceedings., This book constitutes the post-conference proceedings of the First International Workshop on Value Engineering in Artificial Intelligence, VALE 2023, held in Krakow, Poland, on September 30, 2023., In this paper we show how to approach the engineering of values in social human-AI systems by looking into the interplay of three notions: autonomy, governance and value (AGV). We propose a particular characterisation of the Value Alignment Problem based on this approach. We use an example to illustrate how the values can be engineered in order to solve the VAP in this example. Based on these elements we advance some arguments in favour of framing a theory of values that may apply to the governance of social systems that involve artificial autonomous entities as well as humans., Authors wish to acknowledge the contributions and fruitful discussions with Mark d’Inverno, Julian Padget and Harko Verhagen. Research for his paper is supported by EU (Horizon-EIC-2021-Pathfinderchallenges-01) Project VALAWAI 101070930; the EU (NextGenerationEU/PRTR program) and the Spanish (MCIN/AEI-10.13039-501100011033 program) project VAE TED2021-131295B-C31; and CSIC’s (Bilateral Collaboration Initiative i-LINK-TEC) project DESAFIA2030 BILTC22005., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Optimising team dynamics: The role of AI in enhancing challenge-based learning participation experience and outcomes

  • Georgara, Athina
  • Santolini, Marc
  • Kokshagina, Olga
  • Jacinta Haux, Camila Justine
  • Jacobs, Desmé
  • Biwott, Gloria
  • Correa, Marcela
  • Sierra, Carles
  • Fernandez-Marquez, Jose Luis
  • Rodríguez-Aguilar, Juan Antonio
The approach of engaging students with real-world challenges to enhance collaboration and problem-solving has attracted significant interest from scholars and practitioners across diverse disciplines. Often called Challenge-Based Learning (CBL), this educational approach emphasises developing collaborative and problem-solving skills, with significant learning occurring within team settings. Prior studies highlight the influence of team composition on the efficacy of learning outcomes, pointing out that factors such as gender diversity, personality trait diversity, and a wide range of skills affect team dynamics and performance. Despite these insights, the practical organisation of these teams remains a challenge, often reliant on ad-hoc methods driven primarily by the nature of the setting at hand. Importantly, CBL is typically assessed through the final product, neglecting the impact of CBL on how the participants experience the overall process. That is, CBL is usually considered effective if the outcome is of high quality, ignoring participants' experience and participation quality. This study investigates the potential of an Artificial Intelligence team composition algorithm to improve participation quality and outcomes in collaborative CBL environments., This research was conducted under the YOMA OR project (OPE02570). Athina Georgara was supported by the Generalitat de Catalunya under grant 2019DI17, and the VALAWAI project (HE-101070930). Marc Santolini and Olga Koshagina were partly supported by the French Agence Nationale de la Recherche (ANR), under grant agreement ANR-21-CE38-0002. Jose Luis Fernandez-Marquez was supported by ALBATROSS EU project funded by the European Union under G.A. Nº 101137895. Juan A. Rodriguez-Aguilar was funded by projects ACISUD (PID2022-136787NB-I00), VAE (TED2021-131295B-C31), and VALAWAI (HE-101070930)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

From the Pascal Wager to Value Engineering: A Glance at AI Risks and How to Address Them

  • Noriega, Pablo
  • Casanovas, Pompeu
Second International Workshop, VALE 2024, Santiago de Compostela, Spain, October 19–24, 2024, Revised Selected Papers., Conference paper., There is widespread awareness of the potential undesirable consequences of AI. In this paper we explore the grounds of such concerns from the perspective of risk management. We propose a decomposition of AI risk into three sorts of risk (inertial, disruptive, fundamental) that can be approached in different but complementary ways. From this differentiation we advocate for Value Engineering as a pertinent approach to address fundamental AI risk., We wish to thank Jean-Gabriel Ganascia, Ramon L´opez de M´antaras, Mustafa Hashmi, Ho-Pun Lam, and Louis de Koker for their helpful comments. Research for this paper is supported by EU (Horizon-EIC-2021-Pathfinder challenges-01) Project VALAWAI 101070930; the EU Next Generation EU/PRTR program; the Spanish (MCIN/AEI) project VAE TED2021-131295B-C31, CSIC’s (Bilateral Collaboration Initiative i-LINK-TEC) project DESAFIA2030 BILTC22005 and the IDT Research Group of Excellence UAB-CSIC (SGR 00532 2022-24)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

A General Approach for Computing a Consensus in Group Decision Making That Integrates Multiple Ethical Principles

  • Salas-Molina, Francisco
  • Bistaffa, Filippo
  • Rodríguez-Aguilar, Juan Antonio
20 pages, 1 table, 1 figure, We tackle the problem of computing a consensus according to multiple ethical principles – which can include, for example, the principle of maximum freedom associated with the Benthamite doctrine and the principle of maximum fairness associated with the Rawlsian principles – among the preferences of different individuals in the context of Group Decision-Making (GDM). More formally, we put forward a novel formalisation of the above-mentioned problem based on a multi- -norm approximation problem that aims at minimising multiple -metric distance functions, where each parameter represents a given ethical principle. Our contribution incurs obvious benefits from a social-choice perspective. Firstly, our approach significantly generalises state-of-the-art approaches that were limited to only two ethical principles ( , for maximum freedom, and , for maximum fairness). Secondly, our experimental results considering an established test case demonstrate that our approach is capable, thanks to a novel re-weighting scheme, to compute a multi-norm consensus that takes into account each ethical principle in a balanced way, in contrast with state-of-the-art approaches that were heavily biased towards the ethical principle., Research supported by projects: CI-SUSTAIN, Spain (PID2019-104156GB-I00); TAILOR, Spain (H2020-952215); 2021 SGR 00754 funded by Generalitat de Catalunya, Spain; VAE TED2021-131295B-C31, funded by MCIN/AEI, Spain/10.13 039/501100011033 and NextGenerationEU/ PRTR, Spain; and VALAWAI, Spain (Horizon Europe #101070930). Funding for open access charge: CRUE-Universitat Politècnica de València., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

A multidimensional review of the cash management problem

  • Salas-Molina, Francisco
  • Rodríguez-Aguilar, Juan Antonio
  • Guillén, Montserrat
In this paper, we summarize and analyze the relevant research on the cash management problem appearing in the literature. First, we identify the main dimensions of the cash management problem. Next, we review the most relevant contributions in this field and present a multidimensional analysis of these contributions, according to the dimensions of the problem. From this analysis, several open research questions are highlighted., Research supported by projects: Crowd4SDG (H2020-872944); CI-SUSTAIN (PID 2019-104156GB-I00); TAILOR (H2020-952215); TED2021-131295B-C31; TED2021-130187B-I00; PID2019-105986GB-C21 (funded by MCIN/AEI /10.13039/501100011033 and by the “European Union NextGeneration EU/PRTR”); VALAWAI (funded by the European Commission under Grant 101070930); 2021 SGR 00299; 2021 SGR 00754., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

A new bargaining solution for finite offer spaces

  • De Jonge, Dave
[Data Availability] The datasets analyzed during the current study are publicly available with the Genius framework: https://ii.tudelft.nl/genius/, The bargaining problem deals with the question of how far a negotiating agent should concede to its opponent. Classical solutions to this problem, such as the Nash bargaining solution (NBS), are based on the assumption that the set of possible negotiation outcomes forms a continuous space. Recently, however, we proposed a new solution to this problem for scenarios with finite offer spaces de Jonge and Zhang (Auton Agents Multi-Agent Syst 34(1):1–41, 2020). Our idea was to model the bargaining problem as a normal-form game, which we called the concession game, and then pick one of its Nash equilibria as the solution. Unfortunately, however, this game in general has multiple Nash equilibria and it was not clear which of them should be picked. In this paper we fill this gap by defining a new solution to the general problem of how to choose between multiple Nash equilibria, for arbitrary 2-player normal-form games. This solution is based on the assumption that the agent will play either ‘side’ of the game (i.e. as row-player or as column-player) equally often, or with equal probability. We then apply this to the concession game, which ties up the loose ends of our previous work and results in a proper, well-defined, solution to the bargaining problem. The striking conclusion, is that for rational and purely self-interested agents, in most cases the optimal strategy is to agree to the deal that maximizes the sum of the agents’ utilities and not the product of their utilities as the NBS prescribes., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work was supported by a Juan de la Cierva - Incorporación research grant from the Spanish Ministry of Science and Innovation (IJC2018-036443-I) and by Grant no. TED2021-131295B-C31 funded by MCIN/AEI /10.13039/501100011033 and by the European Union NextGenerationEU/PRTR., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Building rankings encompassing multiple criteria to support qualitative decision-making

  • Serramia, Marc
  • López-Sánchez, Maite
  • Moretti, Stefano
  • Rodríguez-Aguilar, Juan Antonio
Decision makers are commonly challenged with comparing, and ultimately ranking, elements with regards to the degree to which they satisfy multiple criteria and in terms of their own preferences. This calls for a new decision making framework, which we formally present here. Within such a framework, we present multi-criteria lex-cel: a new method for ranking single elements. Furthermore, we formally establish that our contributions generalise recent results in the social choice literature. We also illustrate our contributions through a case study that poses an ethical decision-making problem., Research supported by projects: COREDEM (H2020-785907); Crowd4SDG (H2020-872944); CI-SUSTAIN (PID2019-104156GB-I00); TAILOR (H2020-952215); COMRDI18-1-0010-02; 2021 SGR 00754; 2021 SGR 000313; THEMIS (ANR-20-CE23-0018); “FairTransNLP-Language: Analysing toxicity and stereotypes in language for unbiased, fair and transparent systems” (PID2021-124361OB-C33) funded by MCIN/AEI/10.13039/501100011033/FEDER,UE; “AUTODEMO: The stealth side of participatory democracy: process preferences towards automated decision-making” code SR21-00329; “VAE: Value-Awareness Engineering” TED2021-131295B-C31 funded by MCIN/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR; and VALAWAI project, funded by the European Commission under Grant #101070930., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Combining theory of mind and abductive reasoning in agent-oriented programming

  • Montes, Nieves
  • Luck, Michael
  • Osman, Nardine
  • Rodrigues, Odinaldo
  • Sierra, Carles
[Availability of data and materials] All the code accompanying this work can be freely accessed https://github.com/nmontesg/tomabd. All the simulation data generated is freely available https://drive.google.com/file/d/1W66eZD-t-5YeQ4M4SHmdBuX60wRu1e0k/view?usp=sharing., This paper presents a novel model, called TomAbd, that endows autonomous agents with Theory of Mind capabilities. TomAbd agents are able to simulate the perspective of the world that their peers have and reason from their perspective. Furthermore, TomAbd agents can reason from the perspective of others down to an arbitrary level of recursion, using Theory of Mind of order. By combining the previous capability with abductive reasoning, TomAbd agents can infer the beliefs that others were relying upon to select their actions, hence putting them in a more informed position when it comes to their own decision-making. We have tested the TomAbd model in the challenging domain of Hanabi, a game characterised by cooperation and imperfect information. Our results show that the abilities granted by the TomAbd model boost the performance of the team along a variety of metrics, including final score, efficiency of communication, and uncertainty reduction., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. N. Montes, N. Osmain and C. Sierra acknowledge funding from the Spanish funded VAE project (#TED2021-131295B-C31), the EU VALAWAI project (HORIZON #101070930), and the EU TAILOR project (H2020 #952215)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

The AI4Citizen pilot: Pipelining AI-based technologies to support school-work alternation programmes

  • Georgara, Athina
  • Kazhamiakin, Raman
  • Mich, Ornella
  • Palmero Aprosio, Alessio
  • Pazzaglia, Jean Christoph
  • Rodríguez-Aguilar, Juan Antonio
  • Sierra, Carles
[Data availability] The data used in the experiment Section 7.3.2 is available from the corresponding author upon request. The students’ data we use in the rest of the experiments cannot be disclosed due to GDPR issues., The School-Work Alternation (SWA) programme was developed (under a European Commission call) to bridge the gaps and establish a well-tuned partnership between education and the job market. This work details the development of the AI4Citizen pilot, an AI software suite designed to support the SWA programme. The AI4Citizen pilot, developed within the H2020 AI4EU project, offers AI tools to automate and enhance the current SWA process. At the same time, the AI4Citizen pilot offers novel tools to support the complex problem of allocating student teams to internship programs, promoting collaborative learning and teamwork skills acquisition. Notably, the AI4Citizen pilot corresponds to a pipeline of AI tools, integrating existing and novel technologies. Our exhaustive empirical analysis confirms that the AI4Citizen pilot can alleviate the difficulties of current processes in the SWA, and therefore it is ready for real-world deployment., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. The research leading to these results received funding from project AI4EU (H2020-825619). Athina Georgara has received funding from Generalitat de Catalunya under Grant 2019DI17; and has received research support from company Enzyme Advising Group. Furthermore, partial financial support was received from projects: CI-SUSTAIN (PID2019-104156GB-I00); Crowd4SDG (H2020-872944); TAILOR (H2020-952215); VAE (TED2021-131295B-C31), funded by MCIN/AEI /10.13039/501100011033 and by the “European Union NextGenerationEU/PRTR; VALAWAI (Horizon Europe #101070930); and 2021 SGR 00754., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Encoding Ethics to Compute Value-Aligned Norms

  • Serramia, Marc
  • Rodríguez-Soto, Manel
  • López-Sánchez, Maite
  • Rodríguez-Aguilar, Juan Antonio
  • Bistaffa, Filippo
  • Boddington, Paula
  • Wooldridge, Michael
  • Ansotegui, Carlos
Norms have been widely enacted in human and agent societies to regulate individuals’ actions. However, although legislators may have ethics in mind when establishing norms, moral values are only sometimes explicitly considered. This paper advances the state of the art by providing a method for selecting the norms to enact within a society that best aligns with the moral values of such a society. Our approach to aligning norms and values is grounded in the ethics literature. Specifically, from the literature’s study of the relations between norms, actions, and values, we formally define how actions and values relate through the so-called value judgment function and how norms and values relate through the so-called norm promotion function. We show that both functions provide the means to compute value alignment for a set of norms. Moreover, we detail how to cast our decision-making problem as an optimisation problem: finding the norms that maximise value alignment. We also show how to solve our problem using off-the-shelf optimisation tools. Finally, we illustrate our approach with a specific case study on the European Value Study., This work has been funded by projects VAE (TED2021-131295B-C31), VALAWAI (HE-101070930), LOGISTAR (H2020-769142), AI4EU (H2020-825619), Crowd4SDG (H2020-872944), ACISUD (PID2022-136787NB-I00), COREDEM (H2020-785907), Fairtrans (PID2021-124361OB-C33), AUTODEMO (SR21-00329), TAILOR (H2020-952215), assist-Decidim (23S02974-001), 2021 SGR 00313, and 2021 SGR 00754. F. Bistaffa was supported by the H2020-MSCAIF- 2016 HPA4CF project. The authors have no competing interests to declare that are relevant to the content of this article. Maite Lopez-Sanchez belongs to the WAI research group (University of Barcelona) an associated unit to CSIC through the IIIA., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

A Conversational Agent That Learns to be Aligned With the Moral Value of Respect

  • Roselló-Marín, Eric
  • Rodríguez, Inmaculada
  • López-Sánchez, Maite
  • Rodríguez-Soto, Manel
  • Rodríguez-Aguilar, Juan Antonio
Human-Aware Artificial Intelligence Special Issue., Videogame developers typically conduct user experience surveys to gather feedback from users once they have played. Nevertheless, as users may not recall all the details once finished, we propose an ethical conversational agent that respectfully conducts the survey during gameplay. To achieve this without hindering user’s engagement, we resort to reinforcement learning and an ethical embedding algorithm. Specifically, we transform the learning environment so that it guarantees that the agent learns to be respectful (i.e. aligned with the moral value of respect) while pursuing its individual objective of eliciting as much feedback information as possible. When applying this approach to a simple videogame, our comparative tests between the two agents (ethical and unethical) empirically demonstrate that endowing a survey-oriented conversational agent with this moral value of respect avoids disturbing user’s engagement while still pursuing its individual objective, which is to gather as much information as possible., The author(s) received no financial support for the research, authorship, and/or publication of this article., This research was partially supported by projects VAE TED2021-131295B-C31, funded by MCIN/AEI/10.13 039/501100011033 and NextGenerationEU/PRTR, VALAWAI (Horizon Europe #101070930), ACISUD (PID2022-136787NB-I00 funded by MICIU/AEI/10.13039/501100011033), AUTODEMO (SR21-00329) by Fundación La Caixa, Crowd4SDG (H2020-872944), COREDEM (H2020-785907) and FairTransNLP-Language (PID2021-124361OB-C33). Maite Lopez-Sanchez and Inmaculada Rodriguez belong to the WAI research group (University of Barcelona) associated unit to CSIC by IIIA. Rodriguez-Aguilar is also supported by the ‘YOMA Operational Research’ project funded by the Botnar Foundation., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Inteligencia artificial y confianza a través de una comunicación científica comprensible: un compromiso ético con la ciudadanía, Artificial intelligence and trust through understandable scientific communication: an ethical engagement with the public

  • Terrones Rodríguez, Antonio Luis
[EN] The enormous potential of the development of artificial intelligence (AI) can be seen in its increased presence in various environments to offer support for activities that stand out for a high level of complexity. In this sense, institutions such as the European Commission (EC) warn of the need to cultivate trust in AI to guarantee a governance that places the human being at the center of its concerns and benefits. A culture that implies a commitment to strengthening the communications skills of scientific experts as a strategy to promote citizens’ understanding and participation. A scientific communication concerned with the ethical principle of explainability implies valuing interaction with citizens to recognize their testimonies and life experiences, incorporating these into research and innovation processes, [ES] El enorme potencial del desarrollo de la inteligencia artificial (IA) puede observarse en un aumento de su presencia en diversos entornos para ofrecer soporte en actividades que destacan por un alto nivel de complejidad. En ese sentido, instituciones como la Comisión Europea (CE) advierten la necesidad de promover un cultivo de la confianza en la IA para garantizar una gobernanza que sitúe al ser humano en el centro de sus preocupaciones y beneficios. Un cultivo que implica un compromiso con el fortalecimiento de la habilidad comunicativa de los expertos científicos como una estrategia para favorecer la comprensión ciudadana y su participación. Una comunicación científica preocupada por el principio ético de la explicabilidad implica poner en valor la interacción con la ciudadanía para reconocer sus testimonios y experiencias de vida, incorporándolos en los procesos de investigación e innovación., Esta investigación cuenta con la financiación del Ministerio de Universidades del Gobierno de España y la Unión Europea (Next Genera-tion EU) en el marco de las Ayudas Margarita Salas para la formación de jóvenes doctores del programa de recualificación del sistema universitario español. Además, esta publicación es parte del proyecto TED2021-131295B-C31, Ingeniería de Valores de IA-VAE, financiado por MCIN/AEI/10.13039/501100011033 y la Unión Europea Next Generation EU/PRTR, Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Ética, diálogo y robótica para la sostenibilidad en el contexto de la agenda 2030, Ethics, dialogue and robotics for sustainability in the context of the 2030 Agenda

  • Terrones Rodríguez, Antonio Luis
  • Aparicio Payá, Manuel
[EN] We live in a context marked by the global environmental crisis, but also by emerging techno-scientific developments that are going to profoundly transform social life. In this text we will focus on the ethical perspective of ethical of confluence between ecological issues and innovation developments in the field of robotics. An ethical approach to the implementation of robotics, within the framework of the 2030 Agenda, requires a commitment to the environment supported by a model of social innovation in the wake of the five-fold helix. We defend the idea that an ethical robotics committed to the environment requires a constructive culture of dialogue among those affected by this activity, [ES] Vivimos en un contexto marcado por la crisis medioambiental mundial, aunque también por los desarrollos tecnocientíficos emergentes que van a transformar profundamente la vida social. En este trabajo nos centraremos en la perspectiva ética de confluencia entre la problemática ecológica y los desarrollos de innovación en el campo de la robótica. Un enfoque ético para la implementación de la robótica en el marco de la Agenda 2030, exige un compromiso con el medio ambiente apoyado en un modelo de innovación social en la estela de la quíntuple hélice. Defendemos que una robótica ética comprometida con el medio ambiente requiere una cultura constructiva de diálogo entre los afectados por esta actividad, Este trabajo se enmarca en el proyecto europeo INBOTS (780073) del Programa H2020. Cuenta con la financiación del Ministerio de Universidades del Gobierno de España y la Unión Europea (Next Generation EU) en el marco de las Ayudas Margarita Salas para la formación de jóvenes doctores del programa de recualificación del sistema universitario español. Asimismo, se integra en el Proyecto TED2021-131295B-C31 financiado por MCIN/AEI/10.13039/501100011033 y por la Unión Europea NextGenerationEU/PRTR, Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Otros (Other)

An Approximate Embedding for Designing Ethical Reinforcement Learning Environments

  • Mayoral-Macau, Arnau
  • Rodríguez-Soto, Manel
  • Marchesini, Enrico
  • Sánchez-Fibla, Martí
  • López-Sánchez, Maite
  • Rodríguez-Aguilar, Juan Antonio
  • Farinelli, Alessandro
28th European Conference on Artificial Intelligence, 25-30 October 2025, Bologna, Italy – Including 14th Conference on Prestigious Applications of Intelligent Systems (PAIS 2025). Organized by the European Association for Artificial Intelligence (EurAI) and the Italian Association for Artificial Intelligence (AIxIA). Hosted by the University of Bologna., This publication presents the proceedings of ECAI-2025, the 28th European Conference on Artificial Intelligence, held from 25 to 30 October 2025 in Bologna, Italy. Originally organized as a biennial event, ECAI is now held annually, and continuing a long-standing tradition, this edition of ECAI was held alongside the 14th Conference on Prestigious Applications of Intelligent Systems (PAIS). Since its inception in 1974, ECAI has become the premier venue for presenting AI research in Europe, where researchers and practitioners gather to discuss the latest trends and challenges and showcase innovative applications and advances in AI technology. A record total of 3,266 abstract submissions were received for this year’s main track, 2,667 of which were followed by full papers (acceptance rate 23%), together with 81 submissions to the demo track (acceptance rate 33%). Topics cover the full breadth of AI, with many papers exploring interdisciplinary themes. PAIS, an important forum for the sharing of insights into the applicability, development, and deployment of intelligent systems, received 82 submissions (acceptance rate 45%). As well as the papers presented at ECAI and PAIS, this volume also includes eight invited papers on Frontiers in AI corresponding to short invited talks highlighting important new results, techniques, and trends, and the abstracts of 15 recently published papers not previously included in an archival proceedings. This volume of proceedings will be of interest to all those working in the field of Artificial Intelligence., Conference paper., This paper introduces the Approximate Ethical Embedding Process, an algorithm for automating the design of ethical environments for learning agents. Our algorithm helps build environments wherein multiple agents learn policies that align with an ethical (moral) value while simultaneously pursuing their individual objectives. Therefore, we contribute to endowing environment designers with algorithmic tools for building ethical environments. We demonstrate the ethical design process for two different settings of an environment where agents have to adhere to beneficence to promote the collective survival of the population. Our experiments show that our approximate embedding process successfully generates environments that incentivise the learning of value-aligned policies., The research presented in this paper was supported by the EU-funded VALAWAI (# 101070930) project, and the Spanish-funded VAE (# TED2021-131295B-C31), EMOROBCARE (# IASOMMA2024), and ACISUD (PID2022-136787NB-I00) projects., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Multi-objective reinforcement learning for provably incentivising alignment with value systems

  • Rodríguez-Soto, Manel
  • Rădulescu, Roxana
  • Bistaffa, Filippo
  • Ricart, Oriol
  • Mayoral-Macau, Arnau
  • López-Sánchez, Maite
  • Rodríguez-Aguilar, Juan Antonio
  • Nowé, Ann
This paper addresses the problem of ensuring that autonomous learning agents align with multiple moral values. Specifically, we present the theoretical principles and algorithmic tools necessary for creating an environment where we ensure that the agent learns a behaviour aligned with multiple moral values while striving to achieve its individual objective. To address this value alignment problem, we adopt the Multi-Objective Reinforcement Learning framework and propose a novel algorithm that combines techniques from Multi-Objective Reinforcement Learning and Linear Programming. In addition, we illustrate our value alignment process with an example involving an autonomous vehicle. Here, we demonstrate that the agent learns to behave in alignment with the ethical values of safety, achievement, and comfort, with achievement representing the agent’s individual objective. Such ethical behaviour differs depending on the ordering between values. We also use a synthetic multi-objective environment to evaluate the computational costs of guaranteeing ethical learning as the number of values increases., This research project was made possible through the access granted by the Galician Supercomputing Center (CESGA) to its supercomputing infrastructure. The supercomputer FinisTerrae III and its permanent data storage system have been funded by the Spanish Ministry of Science and Innovation, the Galician Government, and the European Regional Development Fund (ERDF). Funding sources The research presented in this paper was supported by the EU-funded VALAWAI (# 101070930) project, the Spanish-funded EVASAI (# PID2024-158227NB-C31), ANNOTATE-SenSIT (# PID2024-156022OB-C33), VAE (# TED2021-131295B-C31), EMOROBCARE (# IASOMMA2024) and Rhymas (# PID2020-113594RB-100) projects. This work was supported by grant PID2022-136787NB-I00 funded by MCIN/AEI/10.13039/501100011033. It was also funded by GUARDEN (101060693), Fairtrans (PID2021-124361OB-C33), AUTODEMO (SR21-00329), TAILOR (H2020-952215), PrepParticip2.0 ( 24S03545-001), grants 2021 SGR 00313 and 2021 SGR 00754. Maite Lopez-Sanchez belongs to the WAI research group (University of Barcelona) associated unit to CSIC by IIIA. Roxana Rădulescu was partly supported by the Research Foundation Flanders (FWO), grant number 1286223N. This research was partially supported by funding from the Flemish Government under the “Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen” program and by the FWO, grant number G062819N., Peer reviewed