Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 21
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Aggregating Value Systems for Decision Support

  • Lera-Leri, Roger X.
  • Liscio, Enrico
  • Bistaffa, Filippo
  • Jonker, Catholijn M.
  • Lopez-Sanchez, Maite
  • Murukannaiah, Pradeep K.
  • Rodriguez-Aguilar, Juan A.
  • Salas-Molina, Francisco|||0000-0002-1168-7931
[EN] We adopt an emerging and prominent vision of human -centred Artificial Intelligence that requires building trustworthy intelligent systems. Such systems should be capable of dealing with the challenges of an interconnected, globalised world by handling plurality and by abiding by human values. Within this vision, pluralistic value alignment is a core problem for AI- that is, the challenge of creating AI systems that align with a set of diverse individual value systems. So far, most literature on value alignment has considered alignment to a single value system. To address this research gap, we propose a novel method for estimating and aggregating multiple individual value systems. We rely on recent results in the social choice literature and formalise the value system aggregation problem as an optimisation problem. We then cast this problem as an l(p)-regression problem. Doing so provides a principled and general theoretical framework to model and solve the aggregation problem. Our aggregation method allows us to consider a range of ethical principles, from utilitarian (maximum utility) to egalitarian (maximum fairness). We illustrate the aggregation of value systems by considering real -world data from two case studies: the Participatory Value Evaluation process and the European Values Study. Our experimental evaluation shows how different consensus value systems can be obtained depending on the ethical principle of choice, leading to practical insights for a decision -maker on how to perform value system aggregation., The authors were supported by the research projects ACISUD (PID2022-136787NB-I00) , VAE (TED2021-131295B-C31) , VALAWAI (HE-101070930) , Fairtrans (PID2021-124361OB-C33) , Yoma OR (OPE02570) , TAILOR (952215) , and the Hybrid Intelligence Center, a 10-year programme funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research. Maite Lopez-Sanchez belongs to the WAI research group (University of Barcelona) , an associated unit to CSIC through the IIIA.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

A multidimensional review of the cash management problem

  • Salas-Molina, Francisco|||0000-0002-1168-7931
  • Rodriguez-Aguilar, Juan A
  • Guillen, Montserrat
[EN] In this paper, we summarize and analyze the relevant research on the cash management problem appearing in the literature. First, we identify the main dimensions of the cash management problem. Next, we review the most relevant contributions in this field and present a multidimensional analysis of these contributions, according to the dimensions of the problem. From this analysis, several open research questions are highlighted., Research supported by projects: Crowd4SDG (H2020872944); CISUSTAIN (PID 2019104156GBI00); TAILOR (H2020952215); TED2021131295BC31; TED2021130187BI00; PID2019105986GBC21 (funded by MCIN/AEI /10.13039/501100011033 and by the European Union NextGeneration EU/PRTR ); VALAWAI (funded by the European Commission under Grant 101070930); 2021 SGR 00299; 2021 SGR 00754.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Aggregating value systems for decision support

  • Lera-Leri, Roger X.
  • Liscio, Enrico
  • Bistaffa, Filippo
  • Jonker, Catholijn M.
  • López-Sánchez, Maite
  • Murukannaiah, Pradeep K.
  • Rodríguez-Aguilar, Juan Antonio
  • Salas-Molina, Francisco
We adopt an emerging and prominent vision of human-centred Artificial Intelligence that requires building trustworthy intelligent systems. Such systems should be capable of dealing with the challenges of an interconnected, globalised world by handling plurality and by abiding by human values. Within this vision, pluralistic value alignment is a core problem for AI– that is, the challenge of creating AI systems that align with a set of diverse individual value systems. So far, most literature on value alignment has considered alignment to a single value system. To address this research gap, we propose a novel method for estimating and aggregating multiple individual value systems. We rely on recent results in the social choice literature and formalise the value system aggregation problem as an optimisation problem. We then cast this problem as an ℓp-regression problem. Doing so provides a principled and general theoretical framework to model and solve the aggregation problem. Our aggregation method allows us to consider a range of ethical principles, from utilitarian (maximum utility) to egalitarian (maximum fairness). We illustrate the aggregation of value systems by considering real-world data from two case studies: the Participatory Value Evaluation process and the European Values Study. Our experimental evaluation shows how different consensus value systems can be obtained depending on the ethical principle of choice, leading to practical insights for a decision-maker on how to perform value system aggregation., The authors were supported by the research projects ACISUD (PID2022-136787NB-I00), VAE (TED2021-131295B-C31), VALAWAI (HE-101070930), Fairtrans (PID2021-124361OB-C33), Yoma OR (OPE02570), TAILOR (952215), and the Hybrid Intelligence Center, a 10-year programme funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research ., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Otros (Other)

Can Interpretability Layouts Influence Human Perception of Offensive Sentences?

  • Freitas dos Santos, Thiago
  • Osman, Nardine
  • Schorlemmer, Marco
6th International Workshop, EXTRAAMAS 2024, Auckland, New Zealand, May 6–10, 2024, Revised Selected Papers.- Conference proceedings., This paper conducts a user study to assess whether three machine learning (ML) interpretability layouts can influence participants’ views when evaluating sentences containing hate speech, focusing on the “Misogyny” and “Racism” classes. Given the existence of divergent conclusions in the literature, we provide statistical and qualitative analyses of questionnaire responses using the Generalized Additive Model to estimate participants’ ratings, incorporating within-subject and between-subject designs. While our statistical analysis indicates that none of the interpretability layouts significantly influences participants’ views, our qualitative analysis demonstrates the advantages of ML interpretability: 1) triggering participants to provide corrective feedback in case of discrepancies between their views and the model, and 2) providing insights to evaluate a model’s behavior beyond traditional performance metrics., This research was supported by the EU funded VALAWAI (# 101070930) and WeNet (# 823783) projects, the Spanish funded VAE (# TED2021-131295BC31) and Rhymas (# PID2020-113594RB-100) projects, and the Generalitat de Catalunya funded Ajuts a grups de recerca de Catalunya (# 2021 SGR 00754) project., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Otros (Other)

On autonomy, governance and values; an AGV approach to value engineering

  • Noriega, Pablo
  • Plaza, Enric
First International Workshop, VALE 2023, Krakow, Poland, September 30, 2023, Proceedings., This book constitutes the post-conference proceedings of the First International Workshop on Value Engineering in Artificial Intelligence, VALE 2023, held in Krakow, Poland, on September 30, 2023., In this paper we show how to approach the engineering of values in social human-AI systems by looking into the interplay of three notions: autonomy, governance and value (AGV). We propose a particular characterisation of the Value Alignment Problem based on this approach. We use an example to illustrate how the values can be engineered in order to solve the VAP in this example. Based on these elements we advance some arguments in favour of framing a theory of values that may apply to the governance of social systems that involve artificial autonomous entities as well as humans., Authors wish to acknowledge the contributions and fruitful discussions with Mark d’Inverno, Julian Padget and Harko Verhagen. Research for his paper is supported by EU (Horizon-EIC-2021-Pathfinderchallenges-01) Project VALAWAI 101070930; the EU (NextGenerationEU/PRTR program) and the Spanish (MCIN/AEI-10.13039-501100011033 program) project VAE TED2021-131295B-C31; and CSIC’s (Bilateral Collaboration Initiative i-LINK-TEC) project DESAFIA2030 BILTC22005., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

What Would I Do If...? Promoting Understanding in HRI through Real-Time Explanations in the Wild

  • Love, Tamlin
  • Andriella, Antonio
  • Alenyà, Guillem
As robots become more and more integrated in human spaces, it is increasingly important for them to be able to explain their decisions to the people they interact with. These explanations need to be generated automatically and in real-time in response to decisions taken in dynamic and often unstructured environments. However, most research in explainable human-robot interaction only considers explanations (often manually selected) presented in controlled environments. We present an explanation generation method based on counterfactuals and demonstrate its use in an "in-the-wild"experiment using automatically generated and selected explanations of autonomous interactions with real people to assess the effect of these explanations on participants' ability to predict the robot's behaviour in hypothetical scenarios. Our results suggest that explanations aid one's ability to predict the robot's behaviour, but also that the addition of counterfactual statements may add some burden and counteract this beneficial effect., This work was supported by Horizon Europe under the MSCA grant agreement No 101072488 (TRAIL); by the “European Union NextGenerationEU/PRTR” project CHLOE-GRAPH PID2020-118649RB-I00 funded by MCIN/ AEI /10.13039/ 501100011033; by the EU-founded project grant agreement No 101070930 (VALAWAI); and by the Research Council of Norway under the project SECUROPS (INT-NO/0875)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Optimising team dynamics: The role of AI in enhancing challenge-based learning participation experience and outcomes

  • Georgara, Athina
  • Santolini, Marc
  • Kokshagina, Olga
  • Jacinta Haux, Camila Justine
  • Jacobs, Desmé
  • Biwott, Gloria
  • Correa, Marcela
  • Sierra, Carles
  • Fernandez-Marquez, Jose Luis
  • Rodríguez-Aguilar, Juan Antonio
The approach of engaging students with real-world challenges to enhance collaboration and problem-solving has attracted significant interest from scholars and practitioners across diverse disciplines. Often called Challenge-Based Learning (CBL), this educational approach emphasises developing collaborative and problem-solving skills, with significant learning occurring within team settings. Prior studies highlight the influence of team composition on the efficacy of learning outcomes, pointing out that factors such as gender diversity, personality trait diversity, and a wide range of skills affect team dynamics and performance. Despite these insights, the practical organisation of these teams remains a challenge, often reliant on ad-hoc methods driven primarily by the nature of the setting at hand. Importantly, CBL is typically assessed through the final product, neglecting the impact of CBL on how the participants experience the overall process. That is, CBL is usually considered effective if the outcome is of high quality, ignoring participants' experience and participation quality. This study investigates the potential of an Artificial Intelligence team composition algorithm to improve participation quality and outcomes in collaborative CBL environments., This research was conducted under the YOMA OR project (OPE02570). Athina Georgara was supported by the Generalitat de Catalunya under grant 2019DI17, and the VALAWAI project (HE-101070930). Marc Santolini and Olga Koshagina were partly supported by the French Agence Nationale de la Recherche (ANR), under grant agreement ANR-21-CE38-0002. Jose Luis Fernandez-Marquez was supported by ALBATROSS EU project funded by the European Union under G.A. Nº 101137895. Juan A. Rodriguez-Aguilar was funded by projects ACISUD (PID2022-136787NB-I00), VAE (TED2021-131295B-C31), and VALAWAI (HE-101070930)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

From the Pascal Wager to Value Engineering: A Glance at AI Risks and How to Address Them

  • Noriega, Pablo
  • Casanovas, Pompeu
Second International Workshop, VALE 2024, Santiago de Compostela, Spain, October 19–24, 2024, Revised Selected Papers., Conference paper., There is widespread awareness of the potential undesirable consequences of AI. In this paper we explore the grounds of such concerns from the perspective of risk management. We propose a decomposition of AI risk into three sorts of risk (inertial, disruptive, fundamental) that can be approached in different but complementary ways. From this differentiation we advocate for Value Engineering as a pertinent approach to address fundamental AI risk., We wish to thank Jean-Gabriel Ganascia, Ramon L´opez de M´antaras, Mustafa Hashmi, Ho-Pun Lam, and Louis de Koker for their helpful comments. Research for this paper is supported by EU (Horizon-EIC-2021-Pathfinder challenges-01) Project VALAWAI 101070930; the EU Next Generation EU/PRTR program; the Spanish (MCIN/AEI) project VAE TED2021-131295B-C31, CSIC’s (Bilateral Collaboration Initiative i-LINK-TEC) project DESAFIA2030 BILTC22005 and the IDT Research Group of Excellence UAB-CSIC (SGR 00532 2022-24)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

A General Approach for Computing a Consensus in Group Decision Making That Integrates Multiple Ethical Principles

  • Salas-Molina, Francisco
  • Bistaffa, Filippo
  • Rodríguez-Aguilar, Juan Antonio
20 pages, 1 table, 1 figure, We tackle the problem of computing a consensus according to multiple ethical principles – which can include, for example, the principle of maximum freedom associated with the Benthamite doctrine and the principle of maximum fairness associated with the Rawlsian principles – among the preferences of different individuals in the context of Group Decision-Making (GDM). More formally, we put forward a novel formalisation of the above-mentioned problem based on a multi- -norm approximation problem that aims at minimising multiple -metric distance functions, where each parameter represents a given ethical principle. Our contribution incurs obvious benefits from a social-choice perspective. Firstly, our approach significantly generalises state-of-the-art approaches that were limited to only two ethical principles ( , for maximum freedom, and , for maximum fairness). Secondly, our experimental results considering an established test case demonstrate that our approach is capable, thanks to a novel re-weighting scheme, to compute a multi-norm consensus that takes into account each ethical principle in a balanced way, in contrast with state-of-the-art approaches that were heavily biased towards the ethical principle., Research supported by projects: CI-SUSTAIN, Spain (PID2019-104156GB-I00); TAILOR, Spain (H2020-952215); 2021 SGR 00754 funded by Generalitat de Catalunya, Spain; VAE TED2021-131295B-C31, funded by MCIN/AEI, Spain/10.13 039/501100011033 and NextGenerationEU/ PRTR, Spain; and VALAWAI, Spain (Horizon Europe #101070930). Funding for open access charge: CRUE-Universitat Politècnica de València., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

A multidimensional review of the cash management problem

  • Salas-Molina, Francisco
  • Rodríguez-Aguilar, Juan Antonio
  • Guillén, Montserrat
In this paper, we summarize and analyze the relevant research on the cash management problem appearing in the literature. First, we identify the main dimensions of the cash management problem. Next, we review the most relevant contributions in this field and present a multidimensional analysis of these contributions, according to the dimensions of the problem. From this analysis, several open research questions are highlighted., Research supported by projects: Crowd4SDG (H2020-872944); CI-SUSTAIN (PID 2019-104156GB-I00); TAILOR (H2020-952215); TED2021-131295B-C31; TED2021-130187B-I00; PID2019-105986GB-C21 (funded by MCIN/AEI /10.13039/501100011033 and by the “European Union NextGeneration EU/PRTR”); VALAWAI (funded by the European Commission under Grant 101070930); 2021 SGR 00299; 2021 SGR 00754., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Building rankings encompassing multiple criteria to support qualitative decision-making

  • Serramia, Marc
  • López-Sánchez, Maite
  • Moretti, Stefano
  • Rodríguez-Aguilar, Juan Antonio
Decision makers are commonly challenged with comparing, and ultimately ranking, elements with regards to the degree to which they satisfy multiple criteria and in terms of their own preferences. This calls for a new decision making framework, which we formally present here. Within such a framework, we present multi-criteria lex-cel: a new method for ranking single elements. Furthermore, we formally establish that our contributions generalise recent results in the social choice literature. We also illustrate our contributions through a case study that poses an ethical decision-making problem., Research supported by projects: COREDEM (H2020-785907); Crowd4SDG (H2020-872944); CI-SUSTAIN (PID2019-104156GB-I00); TAILOR (H2020-952215); COMRDI18-1-0010-02; 2021 SGR 00754; 2021 SGR 000313; THEMIS (ANR-20-CE23-0018); “FairTransNLP-Language: Analysing toxicity and stereotypes in language for unbiased, fair and transparent systems” (PID2021-124361OB-C33) funded by MCIN/AEI/10.13039/501100011033/FEDER,UE; “AUTODEMO: The stealth side of participatory democracy: process preferences towards automated decision-making” code SR21-00329; “VAE: Value-Awareness Engineering” TED2021-131295B-C31 funded by MCIN/AEI/10.13039/501100011033 and by the European Union NextGenerationEU/PRTR; and VALAWAI project, funded by the European Commission under Grant #101070930., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Combining theory of mind and abductive reasoning in agent-oriented programming

  • Montes, Nieves
  • Luck, Michael
  • Osman, Nardine
  • Rodrigues, Odinaldo
  • Sierra, Carles
[Availability of data and materials] All the code accompanying this work can be freely accessed https://github.com/nmontesg/tomabd. All the simulation data generated is freely available https://drive.google.com/file/d/1W66eZD-t-5YeQ4M4SHmdBuX60wRu1e0k/view?usp=sharing., This paper presents a novel model, called TomAbd, that endows autonomous agents with Theory of Mind capabilities. TomAbd agents are able to simulate the perspective of the world that their peers have and reason from their perspective. Furthermore, TomAbd agents can reason from the perspective of others down to an arbitrary level of recursion, using Theory of Mind of order. By combining the previous capability with abductive reasoning, TomAbd agents can infer the beliefs that others were relying upon to select their actions, hence putting them in a more informed position when it comes to their own decision-making. We have tested the TomAbd model in the challenging domain of Hanabi, a game characterised by cooperation and imperfect information. Our results show that the abilities granted by the TomAbd model boost the performance of the team along a variety of metrics, including final score, efficiency of communication, and uncertainty reduction., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. N. Montes, N. Osmain and C. Sierra acknowledge funding from the Spanish funded VAE project (#TED2021-131295B-C31), the EU VALAWAI project (HORIZON #101070930), and the EU TAILOR project (H2020 #952215)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

The AI4Citizen pilot: Pipelining AI-based technologies to support school-work alternation programmes

  • Georgara, Athina
  • Kazhamiakin, Raman
  • Mich, Ornella
  • Palmero Aprosio, Alessio
  • Pazzaglia, Jean Christoph
  • Rodríguez-Aguilar, Juan Antonio
  • Sierra, Carles
[Data availability] The data used in the experiment Section 7.3.2 is available from the corresponding author upon request. The students’ data we use in the rest of the experiments cannot be disclosed due to GDPR issues., The School-Work Alternation (SWA) programme was developed (under a European Commission call) to bridge the gaps and establish a well-tuned partnership between education and the job market. This work details the development of the AI4Citizen pilot, an AI software suite designed to support the SWA programme. The AI4Citizen pilot, developed within the H2020 AI4EU project, offers AI tools to automate and enhance the current SWA process. At the same time, the AI4Citizen pilot offers novel tools to support the complex problem of allocating student teams to internship programs, promoting collaborative learning and teamwork skills acquisition. Notably, the AI4Citizen pilot corresponds to a pipeline of AI tools, integrating existing and novel technologies. Our exhaustive empirical analysis confirms that the AI4Citizen pilot can alleviate the difficulties of current processes in the SWA, and therefore it is ready for real-world deployment., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. The research leading to these results received funding from project AI4EU (H2020-825619). Athina Georgara has received funding from Generalitat de Catalunya under Grant 2019DI17; and has received research support from company Enzyme Advising Group. Furthermore, partial financial support was received from projects: CI-SUSTAIN (PID2019-104156GB-I00); Crowd4SDG (H2020-872944); TAILOR (H2020-952215); VAE (TED2021-131295B-C31), funded by MCIN/AEI /10.13039/501100011033 and by the “European Union NextGenerationEU/PRTR; VALAWAI (Horizon Europe #101070930); and 2021 SGR 00754., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Encoding Ethics to Compute Value-Aligned Norms

  • Serramia, Marc
  • Rodríguez-Soto, Manel
  • López-Sánchez, Maite
  • Rodríguez-Aguilar, Juan Antonio
  • Bistaffa, Filippo
  • Boddington, Paula
  • Wooldridge, Michael
  • Ansotegui, Carlos
Norms have been widely enacted in human and agent societies to regulate individuals’ actions. However, although legislators may have ethics in mind when establishing norms, moral values are only sometimes explicitly considered. This paper advances the state of the art by providing a method for selecting the norms to enact within a society that best aligns with the moral values of such a society. Our approach to aligning norms and values is grounded in the ethics literature. Specifically, from the literature’s study of the relations between norms, actions, and values, we formally define how actions and values relate through the so-called value judgment function and how norms and values relate through the so-called norm promotion function. We show that both functions provide the means to compute value alignment for a set of norms. Moreover, we detail how to cast our decision-making problem as an optimisation problem: finding the norms that maximise value alignment. We also show how to solve our problem using off-the-shelf optimisation tools. Finally, we illustrate our approach with a specific case study on the European Value Study., This work has been funded by projects VAE (TED2021-131295B-C31), VALAWAI (HE-101070930), LOGISTAR (H2020-769142), AI4EU (H2020-825619), Crowd4SDG (H2020-872944), ACISUD (PID2022-136787NB-I00), COREDEM (H2020-785907), Fairtrans (PID2021-124361OB-C33), AUTODEMO (SR21-00329), TAILOR (H2020-952215), assist-Decidim (23S02974-001), 2021 SGR 00313, and 2021 SGR 00754. F. Bistaffa was supported by the H2020-MSCAIF- 2016 HPA4CF project. The authors have no competing interests to declare that are relevant to the content of this article. Maite Lopez-Sanchez belongs to the WAI research group (University of Barcelona) an associated unit to CSIC through the IIIA., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

A Conversational Agent That Learns to be Aligned With the Moral Value of Respect

  • Roselló-Marín, Eric
  • Rodríguez, Inmaculada
  • López-Sánchez, Maite
  • Rodríguez-Soto, Manel
  • Rodríguez-Aguilar, Juan Antonio
Human-Aware Artificial Intelligence Special Issue., Videogame developers typically conduct user experience surveys to gather feedback from users once they have played. Nevertheless, as users may not recall all the details once finished, we propose an ethical conversational agent that respectfully conducts the survey during gameplay. To achieve this without hindering user’s engagement, we resort to reinforcement learning and an ethical embedding algorithm. Specifically, we transform the learning environment so that it guarantees that the agent learns to be respectful (i.e. aligned with the moral value of respect) while pursuing its individual objective of eliciting as much feedback information as possible. When applying this approach to a simple videogame, our comparative tests between the two agents (ethical and unethical) empirically demonstrate that endowing a survey-oriented conversational agent with this moral value of respect avoids disturbing user’s engagement while still pursuing its individual objective, which is to gather as much information as possible., The author(s) received no financial support for the research, authorship, and/or publication of this article., This research was partially supported by projects VAE TED2021-131295B-C31, funded by MCIN/AEI/10.13 039/501100011033 and NextGenerationEU/PRTR, VALAWAI (Horizon Europe #101070930), ACISUD (PID2022-136787NB-I00 funded by MICIU/AEI/10.13039/501100011033), AUTODEMO (SR21-00329) by Fundación La Caixa, Crowd4SDG (H2020-872944), COREDEM (H2020-785907) and FairTransNLP-Language (PID2021-124361OB-C33). Maite Lopez-Sanchez and Inmaculada Rodriguez belong to the WAI research group (University of Barcelona) associated unit to CSIC by IIIA. Rodriguez-Aguilar is also supported by the ‘YOMA Operational Research’ project funded by the Botnar Foundation., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Multi-objective reinforcement learning for designing ethical multi-agent environments

  • Rodríguez-Soto, Manel
  • López-Sánchez, Maite
  • Rodríguez-Aguilar, Juan Antonio
[Data availability] Source code of the environment and algorithms generated and analysed during the current study are available in the following link: https://github.com/Lenmaz/Ethical-Gathering., This paper tackles the open problem of value alignment in multi-agent systems. In particular, we propose an approach to build an ethical environment that guarantees that agents in the system learn a joint ethically-aligned behaviour while pursuing their respective individual objectives. Our contributions are founded in the framework of Multi-Objective Multi-Agent Reinforcement Learning. Firstly, we characterise a family of Multi-Objective Markov Games (MOMGs), the so-called ethical MOMGs, for which we can formally guarantee the learning of ethical behaviours. Secondly, based on our characterisation we specify the process for building single-objective ethical environments that simplify the learning in the multi-agent system. We illustrate our process with an ethical variation of the Gathering Game, where agents manage to compensate social inequalities by learning to behave in alignment with the moral value of beneficence., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature., Work funded by projects VALAWAI (HE-101070930), Crowd4SDG (H2020-872944), TAILOR (H2020-952215), COREDEM (H2020-785907), and 22S01386-001 from Barcelona City Council through the Fundació Solidaritat de la UB. Financial support was also received from grant PID2019-104156GB-I00 funded by MCIN/AEI/10.13039/501100011033. Manel Rodriguez-Soto was funded by the Spanish Government with an FPU grant (ref. FPU18/03387)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Enhancing Robot Assistive Behaviour with Reinforcement Learning and Theory of Mind

  • Andriella, Antonio
  • Falcone, Giovanni
  • Rossi, Silvia
Adaptation to user preferences and the ability to infer and interpret human beliefs and intents, known as the Theory of Mind (ToM), are two critical aspects of effective human-robot collaboration. Despite their importance, very few studies have investigated the impact of adaptive robots with ToM capabilities. This work presents an exploratory comparative study to investigate how social robots simulating ToM capabilities affect user performance and perception. We design a two-tier architecture. The Q-learning agent on the first layer learns the robot’s higher-level behaviour. On the second layer, a heuristic-based ToM infers the user’s intended strategy and is responsible for implementing the robot’s assistance, as well as providing the motivation behind its choice. We conducted a user study in a real-world setting, involving 56 participants who interacted with either an adaptive robot simulating ToM capabilities, or with a robot lacking such capabilities. Our results suggest that participants in the ToM condition performed better, accepted the robot’s assistance more often, and perceived its ability to adapt, predict and detect their intentions to a greater extent. Our preliminary findings could inform future research and pave the way for the design of robot architectures for adaptive behaviour with ToM capabilities, where the choice of action is decoupled from its implementation., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature.This work was partially funded by the Italian Ministry for Universities and Research (MUR) under the complementary actions to the NRRP “Fit4MedRob - Fit for Medical Robotics” Grant (PNC0000007) (S. Rossi), and by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 801342 (Tecniospring INDUSTRY) and by the EU-founded project grant agreement No 101070930 (VALAWAI) (A. Andriella)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Otros (Other)

An Approximate Embedding for Designing Ethical Reinforcement Learning Environments

  • Mayoral-Macau, Arnau
  • Rodríguez-Soto, Manel
  • Marchesini, Enrico
  • Sánchez-Fibla, Martí
  • López-Sánchez, Maite
  • Rodríguez-Aguilar, Juan Antonio
  • Farinelli, Alessandro
28th European Conference on Artificial Intelligence, 25-30 October 2025, Bologna, Italy – Including 14th Conference on Prestigious Applications of Intelligent Systems (PAIS 2025). Organized by the European Association for Artificial Intelligence (EurAI) and the Italian Association for Artificial Intelligence (AIxIA). Hosted by the University of Bologna., This publication presents the proceedings of ECAI-2025, the 28th European Conference on Artificial Intelligence, held from 25 to 30 October 2025 in Bologna, Italy. Originally organized as a biennial event, ECAI is now held annually, and continuing a long-standing tradition, this edition of ECAI was held alongside the 14th Conference on Prestigious Applications of Intelligent Systems (PAIS). Since its inception in 1974, ECAI has become the premier venue for presenting AI research in Europe, where researchers and practitioners gather to discuss the latest trends and challenges and showcase innovative applications and advances in AI technology. A record total of 3,266 abstract submissions were received for this year’s main track, 2,667 of which were followed by full papers (acceptance rate 23%), together with 81 submissions to the demo track (acceptance rate 33%). Topics cover the full breadth of AI, with many papers exploring interdisciplinary themes. PAIS, an important forum for the sharing of insights into the applicability, development, and deployment of intelligent systems, received 82 submissions (acceptance rate 45%). As well as the papers presented at ECAI and PAIS, this volume also includes eight invited papers on Frontiers in AI corresponding to short invited talks highlighting important new results, techniques, and trends, and the abstracts of 15 recently published papers not previously included in an archival proceedings. This volume of proceedings will be of interest to all those working in the field of Artificial Intelligence., Conference paper., This paper introduces the Approximate Ethical Embedding Process, an algorithm for automating the design of ethical environments for learning agents. Our algorithm helps build environments wherein multiple agents learn policies that align with an ethical (moral) value while simultaneously pursuing their individual objectives. Therefore, we contribute to endowing environment designers with algorithmic tools for building ethical environments. We demonstrate the ethical design process for two different settings of an environment where agents have to adhere to beneficence to promote the collective survival of the population. Our experiments show that our approximate embedding process successfully generates environments that incentivise the learning of value-aligned policies., The research presented in this paper was supported by the EU-funded VALAWAI (# 101070930) project, and the Spanish-funded VAE (# TED2021-131295B-C31), EMOROBCARE (# IASOMMA2024), and ACISUD (PID2022-136787NB-I00) projects., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

Multi-objective reinforcement learning for provably incentivising alignment with value systems

  • Rodríguez-Soto, Manel
  • Rădulescu, Roxana
  • Bistaffa, Filippo
  • Ricart, Oriol
  • Mayoral-Macau, Arnau
  • López-Sánchez, Maite
  • Rodríguez-Aguilar, Juan Antonio
  • Nowé, Ann
This paper addresses the problem of ensuring that autonomous learning agents align with multiple moral values. Specifically, we present the theoretical principles and algorithmic tools necessary for creating an environment where we ensure that the agent learns a behaviour aligned with multiple moral values while striving to achieve its individual objective. To address this value alignment problem, we adopt the Multi-Objective Reinforcement Learning framework and propose a novel algorithm that combines techniques from Multi-Objective Reinforcement Learning and Linear Programming. In addition, we illustrate our value alignment process with an example involving an autonomous vehicle. Here, we demonstrate that the agent learns to behave in alignment with the ethical values of safety, achievement, and comfort, with achievement representing the agent’s individual objective. Such ethical behaviour differs depending on the ordering between values. We also use a synthetic multi-objective environment to evaluate the computational costs of guaranteeing ethical learning as the number of values increases., This research project was made possible through the access granted by the Galician Supercomputing Center (CESGA) to its supercomputing infrastructure. The supercomputer FinisTerrae III and its permanent data storage system have been funded by the Spanish Ministry of Science and Innovation, the Galician Government, and the European Regional Development Fund (ERDF). Funding sources The research presented in this paper was supported by the EU-funded VALAWAI (# 101070930) project, the Spanish-funded EVASAI (# PID2024-158227NB-C31), ANNOTATE-SenSIT (# PID2024-156022OB-C33), VAE (# TED2021-131295B-C31), EMOROBCARE (# IASOMMA2024) and Rhymas (# PID2020-113594RB-100) projects. This work was supported by grant PID2022-136787NB-I00 funded by MCIN/AEI/10.13039/501100011033. It was also funded by GUARDEN (101060693), Fairtrans (PID2021-124361OB-C33), AUTODEMO (SR21-00329), TAILOR (H2020-952215), PrepParticip2.0 ( 24S03545-001), grants 2021 SGR 00313 and 2021 SGR 00754. Maite Lopez-Sanchez belongs to the WAI research group (University of Barcelona) associated unit to CSIC by IIIA. Roxana Rădulescu was partly supported by the Research Foundation Flanders (FWO), grant number 1286223N. This research was partially supported by funding from the Flemish Government under the “Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen” program and by the FWO, grant number G062819N., Peer reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Exploring the potential of a robot-assisted frailty assessment system for elderly care

  • Civit Bertran, Aniol|||0009-0000-6255-3362
  • Andriella, Antonio|||0000-0002-6641-6450
  • Antonio, Maite
  • Javierre Garcés, Casimiro
  • Boqué, Concepción
  • Alenyà Ribas, Guillem|||0000-0002-6018-154X
© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Frailty assessment plays a pivotal role in providing older adults care. However, the current process is time- consuming and only measures patients’ completion time for each test. This paper introduces a set of algorithms to be used in robots to autonomously perform frailty assessments. In doing so we aim at reducing therapists’ burden and provide additional frailty-related metrics that can enhance the effectiveness of diagnosis. We conducted a pilot study with 22 elderly participants and compared our system’s performance with that of medical professionals to assess its precision. The results demonstrate that our approach achieved performances close to that of its human counterpart. This research represents an important step forward in the integration of social robotics in healthcare, offering potential benefits for patient care and clinical decision-making., This work was supported by the project ROBIN PLEC2021-007859 funded by MCIN/ AEI /10.13039/501100011033 and by the "European Union NextGenerationEU/PRTR"; by the EU-founded project grant agreement No 101070930 (VALAWAI); and the 23S06141-001 FRAILWATCH project funded by Barcelona Ciencia 2020-2023 Plan., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Exploring the potential of a robot-assisted frailty assessment system for elderly care

  • Civit Bertran, Aniol
  • Andriella, Antonio
  • Antonio, Maite
  • Javierre Garcés, Casimiro
  • Boqué, Concepción
  • Alenyà Ribas, Guillem
© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Frailty assessment plays a pivotal role in providing older adults care. However, the current process is time- consuming and only measures patients’ completion time for each test. This paper introduces a set of algorithms to be used in robots to autonomously perform frailty assessments. In doing so we aim at reducing therapists’ burden and provide additional frailty-related metrics that can enhance the effectiveness of diagnosis. We conducted a pilot study with 22 elderly participants and compared our system’s performance with that of medical professionals to assess its precision. The results demonstrate that our approach achieved performances close to that of its human counterpart. This research represents an important step forward in the integration of social robotics in healthcare, offering potential benefits for patient care and clinical decision-making., This work was supported by the project ROBIN PLEC2021-007859 funded by MCIN/ AEI /10.13039/501100011033 and by the "European Union NextGenerationEU/PRTR"; by the EU-founded project grant agreement No 101070930 (VALAWAI); and the 23S06141-001 FRAILWATCH project funded by Barcelona Ciencia 2020-2023 Plan., Peer Reviewed, Postprint (author's final draft)