Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 35
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Deep reinforcement learning in medicine

  • Jonsson, Anders, 1973-
Reinforcement learning has achieved tremendous success in recent years, notably in complex games such as Atari, Go, and chess. In large part, this success has been made possible by powerful function approximation methods in the form of deep neural networks. The objective of this paper is to introduce the basic concepts of reinforcement learning, explain how reinforcement learning can be effectively combined with deep learning, and explore how deep reinforcement learning could be useful in a medical context., This work is partially funded by the grant TIN2015-67959 of the Spanish Ministry of Science.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Generalized planning as heuristic search

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders, 1973-
Comunicació presentada a ICAPS2021, celebrat del 2 a 13 d'agost de 2021 de manera virtual., Although heuristic search is one of the most successful approaches to classical planning, this planning paradigm does not apply straightforwardly to Generalized Planning (GP). Planning as heuristic search traditionally addresses the computation of sequential plans by searching in a grounded state-space. On the other hand, GP aims at computing algorithm-like plans that can branch and loop, and that generalize to a (possibly infinite) set of planning instances. This paper adapts the {\em planning as heuristic search} paradigm to the particularities of GP and presents the first native heuristic search approach to GP. First, the paper defines a novel GP solution space that is independent of the number of planning instances in a GP problem, and the size of these instances. Second, the paper defines several evaluation and heuristic functions, that do not require grounding, for guiding a combinatorial search in the GP solution space, making it possible to handle state variables with large numerical domains (e.g. integers). Lastly, the paper defines an algorithm for GP called Best-First Generalized Planning (BFGP), that implements a best-first search in the solution space guided by our evaluation/heuristic functions., Javier Segovia-Aguas is supported by TAILOR, a project funded by EU H2020 research and innovation programme no. 952215, an ERC Advanced Grant no. 885107, and grant TIN-2015-67959-P from MINECO, Spain. Sergio Jiménez is supported by the Ramon y Cajal program, RYC-2015-18009, the Spanish MINECO project TIN2017-88476-C2-1-R. Anders Jonsson is partially supported by Spanish grants PID2019-108141GB-I00 and PCIN-2017-082.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

∃-STRIPS: existential quantification in planning and constraint satisfaction

  • Francès, Guillem
  • Geffner, Héctor
Comunicació presentada a la Twenty-Fifth International Joint Conference on Artificial Intelligence, celebrada els dies 9 a 15 de juliol de 2016 a Nova York., Existentially quantified variables in goals and action preconditions are part of the standard PDDL planning language, yet few planners support them, while those that do compile them away at an exponential cost. In this work, we argue that existential variables are an essential feature for representing and reasoning with constraints in planning, and that it is harmful to compile them away or avoid them altogether, since this hides part of the problem structure that can be exploited computationally. We show how to do this by formulating an extension of the standard delete-relaxation heuristics that handles existential variables. While this extension is simple, the consequences for both modeling and computation are important. Furthermore, by allowing existential variables in STRIPS and treating them properly, CSPs can be represented and solved in a direct manner as action-less,fluent-less STRIPS planning problems, something important for problems involving restrictions. In addition, functional fluents in Functional STRIPS can be compiled away with no effect on the structure and informativeness of the resulting heuristic. Experiments are reported comparing our native ∃-STRIPS planner with state-of-the-art STRIPS planners over compiled and propositional encodings, and with a Functional STRIPS planner., This work is partially supported by the M. de Maeztu Programme (MDM-2015-0502) and by grants TIN2015-67959 and CSD2010-00034, MEC, Spain.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Computing programs for generalized planning using a classical planner

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders
Although heuristic search is one of the most successful approaches to classical planning, this planning paradigm does not apply straightforwardly to Generalized Planning (GP). This paper adapts the planning as heuristic search paradigm to the particularities of GP, and presents the first native heuristic search approach to GP. First, the paper defines a program-based solution space for GP that is independent of the number of planning instances in a GP problem, and the size of these instances. Second, the paper defines the BFGP algorithm for GP, that implements a best-first search in our programbased solution space, and that is guided by different evaluation and heuristic functions.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Learning general policies from small examples without supervision

  • Francès, Guillem
  • Bonet, Blai
  • Geffner, Héctor
Generalized planning is concerned with the computation of general policies that solve multiple instances of a planning domain all at once. It has been recently shown that these policies can be computed in two steps: first, a suitable abstraction in the form of a qualitative numerical planning problem (QNP) is learned from sample plans, then the general policies are obtained from the learned QNP using a planner. In this work, we introduce an alternative approach for computing more expressive general policies which does not require sample plans or a QNP planner. The new formulation is very simple and can be cast in terms that are more standard in machine learning: a large but finite pool of features is defined from the predicates in the planning examples using a general grammar, and a small subset of features is sought for separating “good” from “bad” state transitions, and goals from non-goals. The problems of finding such a “separating surface” while labeling the transitions as “good” or “bad” are jointly addressed as a single combinatorial optimization problem expressed as a Weighted Max-SAT problem. The advantage of looking for the simplest policy in the given feature space that solves the given examples, possibly non-optimally, is that many domains have no general, compact policies that are optimal. The approach yields general policies for a number of benchmark domains., This research is partially funded by an ERC Advanced Grant (No 885107), by grant TIN-2015-67959-P from MINECO, Spain, and by the Knut and Alice Wallenberg (KAW) Foundation through the WASP program. H. Geffner is also a Wallenberg Guest Professor at Linköping University, Sweden. G. Francès is partially supported by grant IJC2019-039276-I from MICINN, Spain.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Unsupervised classification of planning instances

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders, 1973-
Comunicació presentada a: The 27th International Conference on Automated Planning and Scheduling, ICAPS 2017, celebrada a Pittsburgh, Estats Units, del 18 al 23 de juny de 2017, In this paper we introduce a novel approach for unsupervised classification of planning instances based on the recent formalism of planning programs. Our approach is inspired by structured prediction in machine learning, which aims at predicting structured information about a given input rather than a scalar value. In our case, each input is an unlabelled classical planning instance, and the associated structured information is the planning program that solves the instance. We describe a method that takes as input a set of planning instances and outputs a set of planning programs, classifying each instance according to the program that solves it. Our results show that automated planning can be successfully used to solve structured unsupervised classification tasks, and invites further exploration of the connection between automated planning and structured prediction., This work is partially supported by grant TIN2015-67959 and the Maria de Maeztu Units of Excellence Programme MDM-2015-0502, MEC, Spain.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Purely declarative action descriptions are overrated: classical planning with simulators

  • Francès, Guillem
  • Ramírez Jávega, Miquel
  • Lipovetzky, Nir
  • Geffner, Héctor
Comunicació presentada a la Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI 2017), celebrada els dies 19 a 25 d'agost a Melbourne, Austràlia., Classical planning is concerned with problems where a goal needs to be reached from a known initial state by doing actions with deterministic, known effects. Classical planners, however, deal only with classical problems that can be expressed in declarative planning languages such as STRIPS or PDDL. This prevents their use on problems that are not easy to model declaratively or whose dynamics are given via simulations. Simulators do not provide a declarative representation of actions, but simply return successor states. The question we address in this paper is: can a planner that has access to the structure of states and goals only, approach the performance of planners that also have access to the structure of actions expressed in PDDL? To answer this, we develop domain-independent, black box planning algorithms that completely ignore action structure, and show that they match the performance of state-of-the-art classical planners on the standard planning benchmarks. Effective black box algorithms open up new possibilities for modeling and for expressing control knowledge, which we also illustrate., G. Francès is supported by the M. de Maeztu Programme (MDM-2015-0502) and H. Geffner by grant TIN2015-67959-P, both from MINECO, Spain. M. Ramirez and N. Lipovetzky have been partially funded by the Australian DST Group.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Solving multiagent planning problems with concurrent conditional effects

  • Furelos Blanco, Daniel
  • Jonsson, Anders, 1973-
Comunicació presentada al 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019 and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, celebrat del 27 de gener a l'1 de febrer de 2019 a Palo Alta, EEUU., In this work we present a novel approach to solving concurrent multiagent planning problems in which several agents act in parallel. Our approach relies on a compilation from concurrent multiagent planning to classical planning, allowing us to use an off-the-shelf classical planner to solve the original multiagent problem. The solution can be directly interpreted as a concurrent plan that satisfies a given set of concurrency constraints, while avoiding the exponential blowup associated with concurrent actions. Our planner is the first to handle action effects that are conditional on what other agents are doing. Theoretically, we show that the compilation is sound and complete. Empirically, we show that our compilation can solve challenging multiagent planning problems that require concurrent actions., This work has been supported by the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502). Anders Jonsson is partially supported by the grants TIN2015-67959 and PCIN-2017-082 of the Spanish Ministry of Science.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Automatic generation of high-level state features for generalized planning

  • Lotinac, Damir
  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders, 1973-
In many domains generalized plans can only/nbe computed if certain high-level state features,/ni.e. features that capture key concepts to accurately/ndistinguish between states and make good decisions,/nare available. In most applications of generalized/nplanning such features are hand-coded by/nan expert. This paper presents a novel method/nto automatically generate high-level state features/nfor solving a generalized planning problem. Our/nmethod extends a compilation of generalized planning/ninto classical planning and integrates the computation/nof generalized plans with the computation/nof features, in the form of conjunctive queries. Experiments/nshow that we generate features for diverse/ngeneralized planning problems and hence,/ncompute generalized plans without providing a/nprior high-level representation of the states. We/nalso bring a new landscape of challenging benchmarks/nto classical planning since our compilation/nnaturally models classification tasks as classical/nplanning problems., This work is partially supported by grant TIN2015-67959 and the/nMaria de Maeztu Units of Excellence Programme MDM-2015-/n0502, MEC, Spain. Sergio Jimenez is partially supported by the /nJuan de la Cierva program funded by the Spanish government.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

General policies, representations, and planning width

  • Bonet, Blai
  • Geffner, Héctor
It has been observed that in many of the benchmark planning domains, atomic goals can be reached with a simple polynomial exploration procedure, called IW, that runs in time exponential in the problem width. Such problems have indeed a bounded width: a width that does not grow with the number of problem variables and is often no greater than two. Yet, while the notion of width has become part of the state- of-the-art planning algorithms like BFWS, there is still no good explanation for why so many benchmark domains have bounded width. In this work, we address this question by relating bounded width and serialized width to ideas of generalized planning, where general policies aim to solve multiple instances of a planning problem all at once. We show that bounded width is a property of planning domains that admit optimal general policies in terms of features that are explicitly or implicitly represented in the domain encoding. The results are extended to the larger class of domains with bounded serialized width where the general policies do not have to be optimal. The study leads also to a new simple, meaningful, and expressive language for specifying domain serializations in the form of policy sketches which can be used for encoding domain control knowledge by hand or for learning it from traces. The use of sketches and the meaning of the theoretical results are all illustrated through a number of examples., The research is partially funded by an ERC Advanced Grant (No 885107), by grant TIN-2015-67959-P from MINECO, Spain, and by the Knut and Alice Wallenberg (KAW) Foundation through the WASP program. H. Geffner is a also Wallenberg Guest Professor at Linkoping University, Sweden.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Generating context-free grammars using classical planning

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders, 1973-
Comunicació presentada a: IJCAI International Joint Conference on Artificial Intelligence, celebrada a Melbourne, Austràlia, del 19 al 25 d'agost de 2017., This paper presents a novel approach for generating Context-Free Grammars (CFGs) from small sets of input strings (a single input string in some cases). Our approach is to compile this task into a classical planning problem whose solutions are sequences of actions that build and validate a CFG compliant with the input strings. In addition, we show that our compilation is suitable for implementing the two canonical tasks for CFGs, string production and string recognition., This work is partially supported by grant TIN2015-67959 and the Maria de Maeztu Units of Excellence Programme MDM-2015-0502, MEC, Spain.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Lifelong control of off-grid microgrid with model-based reinforcement learning

  • Totaro, Simone
  • Boukas, Ioannis
  • Jonsson, Anders, 1973-
  • Cornélusse, Bertrand
Off-grid microgrids are receiving a growing interest for rural electrification purposes in developing countries due to their ability to ensure affordable, sustainable and reliable energy services. Off-grid microgrids rely on renewable energy sources (RES) coupled with storage systems to supply the electrical consumption. The inherent uncertainty introduced by RES as well as the stochastic nature of the electrical demand in rural contexts pose significant challenges to the efficient control of off-grid microgrids throughout their entire life span. In this paper, we address the lifelong control problem of an isolated microgrid. We categorize the set of changes that may occur over its life span in progressive and abrupt changes. We propose a novel model-based reinforcement learning algorithm that is able to address both types of changes. In particular, the proposed algorithm demonstrates generalisation properties, transfer capabilities and better robustness in case of fast-changing system dynamics. The proposed algorithm is compared against a rule-based policy and a model predictive controller with look-ahead. The results show that the trained agent is able to outperform both benchmarks in the lifelong setting where the system dynamics are changing over time., This research is carried out in the framework of the project Dynamically Evolving Long-Term Autonomy (DELTA), a European research project funded under the CHIST-ERA scheme (http://www.chistera.eu/). Anders Jonsson is partially supported by the grants TIN2015-67959 and PCIN-2017-082 of the Spanish Ministry of Science. The authors would like to thank Sergio Balderrama for the provision of measured data from the “El Espino” microgrid in Bolivia and Alessandro Davide Ialongo for the fruitful discussion.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Computing hierarchical finite state controllers with classical planning

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders, 1973-
Finite State Controllers (FSCs) are an effective way to compactly represent sequential plans. By imposing appropriate conditions on transitions, FSCs can also represent generalized plans (plans that solve a range of planning problems from a given domain). In this paper we introduce the concept of hierarchical FSCs for planning by allowing controllers to call other controllers. This call mechanism allows hierarchical FSCs to represent generalized plans more compactly than individual FSCs, to compute controllers in a modular fashion or even more, to compute recursive controllers. The paper introduces a classical planning compilation for computing hierarchical FSCs that solve challenging generalized planning tasks. The compilation takes as input a finite set of classical planning problems from a given domain. The output of the compilation is a single classical planning problem whose solution induces: (1) a hierarchical FSC and (2), the corresponding validation of that controller on the input classical planning problems., This work is partially supported by grant TIN2015-67959 and the Maria de Maeztu Units of Excellence Programme MDM-2015-0502, MEC, Spain. Sergio Jiménez is supported by the RYC15/18009, program funded by the Spanish government.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Hierarchical finite state controllers for generalized planning

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders, 1973-
Finite State Controllers (FSCs) are an effective way/nto represent sequential plans compactly. By imposing/nappropriate conditions on transitions, FSCs/ncan also represent generalized plans that solve a/nrange of planning problems from a given domain./nIn this paper we introduce the concept of hierarchical/nFSCs for planning by allowing controllers/nto call other controllers. We show that hierarchical/nFSCs can represent generalized plans more compactly/nthan individual FSCs. Moreover, our call/nmechanism makes it possible to generate hierarchical/nFSCs in a modular fashion, or even to apply/nrecursion. We also introduce a compilation that/nenables a classical planner to generate hierarchical/nFSCs that solve challenging generalized planning/nproblems. The compilation takes as input a set of/nplanning problems from a given domain and outputs/na single classical planning problem, whose solution/ncorresponds to a hierarchical FSC., This work is partially supported by grant TIN2015-67959 and the/nMaria de Maeztu Units of Excellence Programme MDM-2015-/n0502, MEC, Spain. Sergio Jimenez is partially supported by the /nJuan de la Cierva program funded by the Spanish government.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Constructing hierarchical task models using invariance analysis

  • Jonsson, Anders, 1973-
  • Lotinac, Damir
Comunicació presentada a la biennial European Conference on Artificial Intelligence (ECAI 2016), 22nd European Conference on Artificial Intelligence, celebrada a La Haia (Països Baixos) del 29 d'agost al 2 de septembre de 2016., Hierarchical Task Networks (HTNs) are a common model for encoding knowledge about planning domains in the form of task decompositions. We present a novel algorithm that uses invariant analysis to construct an HTN from the PDDL description of a planning domain and a single representative instance. The algorithm defines two types of composite tasks that interact to achieve the goal of a planning instance. One type of task achieves fluents by traversing invariants in which only one fluent can be true at a time. The other type of task applies a single action, which first involves ensuring that the precondition of the action holds. The resulting HTN can be applied to any instance of the planning domain, and is provably sound. We show that the performance of our algorithm is comparable to algorithms that learn HTNs from examples and use added knowledge., This work is partially supported by the MINECO/FEDER grant TIN2015-67959 and the Maria de Maeztu Units of Excellence Programme MDM-2015-0502, MEC, Spain.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Computing Hierarchical Finite State Controllers with Classical Planning

  • Segovia-Aguas, Javier
  • Jonsson, Anders
  • Jiménez-Celorrio, Sergio|||0000-0003-0561-4880
[EN] Finite State Controllers (FSCs) are an effective way to compactly represent sequential plans. By imposing appropriate conditions on transitions, FSCs can also represent generalized plans (plans that solve a range of planning problems from a given domain). In this paper we introduce the concept of hierarchical FSCs for planning by allowing controllers to call other controllers. This call mechanism allows hierarchical FSCs to represent generalized plans more compactly than individual FSCs, to compute controllers in a modular fashion or even more, to compute recursive controllers. The paper introduces a classical planning compilation for computing hierarchical FSCs that solve challenging generalized planning tasks. The compilation takes as input a finite set of classical planning problems from a given domain. The output of the compilation is a single classical planning problem whose solution induces: (1) a hierarchical FSC and (2), the corresponding validation of that controller on the input classical planning problems., This work is partially supported by grant TIN2015-67959 and the Maria de Maeztu Units of Excellence Programme MDM-2015-0502, MEC, Spain. Sergio Jimenez is supported by the RYC15/18009, program funded by the Spanish government.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Contribución de congreso (ConferenceOutput)

Generalized Planning with Positive and Negative Examples

  • Segovia-Aguas, Javier
  • Jonsson, Anders
  • Jiménez-Celorrio, Sergio|||0000-0003-0561-4880
[EN] Generalized planning aims at computing an algorithm-like structure (generalized plan) that solves a set of multiple planning instances. In this paper we define negative examples for generalized planning as planning instances that must not be solved by a generalized plan. With this regard the paper extends the notion of validation of a generalized plan as the problem of verifying that a given generalized plan solves the set of input positives instances while it fails to solve a given input set of negative examples. This notion of plan validation allows us to define quantitative metrics to asses the generalization capacity of generalized plans. The paper also shows how to incorporate this new notion of plan validation into a compilation for plan synthesis that takes both positive and negative instances as input. Experiments show that incorporating negative examples can accelerate plan synthesis in several domains and leverage quantitative metrics to evaluate the generalization capacity of the synthesized plans., The research leading to these results has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement no. 731761, IMAGINE; and it is partially supported by grant TIN-2015- 67959 and the Maria de Maeztu Units of Excellence Programme MDM-2015-0502, MEC, Spain. Sergio Jimenez is supported by the Ramon y Cajal program, RYC-2015- 18009, the Spanish MINECO project TIN2017-88476-C2- 1-R, and the generalitat valenciana project GV/2019/082. Anders Jonsson is partially supported by the Spanish grants TIN2015-67959 and PCIN-2017-082.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publication Artículo científico (JournalArticle)

Computing programs for generalized planning using a classical planner

  • Segovia-Aguas, Javier
  • Jonsson, Anders
  • Jiménez-Celorrio, Sergio|||0000-0003-0561-4880
[EN] Generalized planning is the task of generating a single solution (a generalized plan) that is valid for multiple planning instances. In this paper we introduce a novel formalism for representing generalized plans that borrows two mechanisms from structured programming: control flow and procedure calls. On one hand, control flow structures allow to compactly represent generalized plans. On the other hand, procedure calls allow to represent hierarchical and recursive solutions as well as to reuse existing generalized plans. The paper also presents a compilation from generalized planning into classical planning which allows us to compute generalized plans with off-the-shelf planners. The compilation can incorporate prior knowledge in the form of auxiliary procedures which expands the applicability of the approach to more challenging tasks. Experiments show that a classical planner using our compilation can compute generalized plans that solve a wide range of generalized planning tasks, including sorting lists of variable size or DFS traversing variable-size binary trees. Additionally the paper presents an extension of the compilation for computing generalized plans when generalization requires a high-level state representation that is not provided a priori. This extension brings a new landscape of benchmarks to classical planning since classification tasks can naturally be modeled as generalized planning tasks, and hence, as classical planning tasks. Finally the paper shows that the compilation can be extended to compute control knowledge for off-the-shelf planners and solve planning instances that are difficult to solve without such additional knowledge., Anders Jonsson is partially supported by the grants TIN2015-67959 and PCIN-2017-082 of the Spanish Ministry of Science. Sergio Jimenez is partially supported by the grants, RYC-2015-18009 and TIN2017-88476-C2-1-R of the Spanish Ministry of Science.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Generalized planning with positive and negative examples

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders
Trabajo presentado en la 34th AAAI Conference on Artificial Intelligence (AAAI), celebrada en New York (Estados Unidos), del 7 al 12 de febrero de 2020, Generalized planning aims at computing an algorithm-like structure (generalized plan) that solves a set of multiple planning instances. In this paper we define negative examples for generalized planning as planning instances that must not be solved by a generalized plan. With this regard the paper extends the notion of validation of a generalized plan as the problem of verifying that a given generalized plan solves the set of input positives instances while it fails to solve a given input set of negative examples. This notion of plan validation allows us to define quantitative metrics to asses the generalization capacity of generalized plans. The paper also shows how to incorporate this new notion of plan validation into a compilation for plan synthesis that takes both positive and negative instances as input. Experiments show that incorporating negative examples can accelerate plan synthesis in several domains and leverage quantitative metrics to evaluate the generalization capacity of the synthesized plans., The research leading to these results has received funding from the European Unions Horizon 2020 research and innovation programme under grant agreement no. 731761, IMAGINE; and it is partially supported by grant TIN-2015- 67959 and the Maria de Maeztu Units of Excellence Programme MDM-2015-0502, MEC, Spain. Sergio Jimenez ´ is supported by the Ramon y Cajal program, RYC-2015- 18009, the Spanish MINECO project TIN2017-88476-C2- 1-R, and the generalitat valenciana project GV/2019/082. Anders Jonsson is partially supported by the Spanish grants TIN2015-67959 and PCIN-2017-082.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Online action recognition

  • Suárez Hernández, Alejandro
  • Segovia-Aguas, Javier
  • Torras, Carme
  • Alenyà, Guillem
Trabajo presentado en 35th AAAI Conference on Artificial Intelligence, celebrada de forma virtual del 2 al 9 de febrero de 2021, Recognition in planning seeks to find agent intentions, goals or activities given a set of observations and a knowledge library (e.g. goal states, plans or domain theories). In this work we introduce the problem of Online Action Recognition. It consists in recognizing, in an open world, the planning action that best explains a partially observable state transition from a knowledge library of first-order STRIPS actions, which is initially empty. We frame this as an optimization problem, and propose two algorithms to address it: Action Unification (AU) and Online Action Recognition through Unification (OARU). The former builds on logic unification and generalizes two input actions using weighted partial MaxSAT. The latter looks for an action within the library that explains an observed transition. If there is such action, it generalizes it making use of AU, building in this way an AU hierarchy. Otherwise, OARU inserts a Trivial Grounded Action (TGA) in the library that explains just that transition. We report results on benchmarks from the International Planning Competition and PDDLGym, where OARU recognizes actions accurately with respect to expert knowledge, and shows real-time performance., The research leading to these results has received funding from the EU H2020 research and innovation programme under grant agreement no.731761, IMAGINE; the Hu- MoUR project TIN2017-90086-R (AEI/FEDER, UE); and AEI through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656). Javier Segovia-Aguas was also partially supported by TAILOR, a project funded by EU H2020 research and innovation programme no. 952215, an ERC Advanced Grant no. 885107, and grant TIN-2015-67959-P from MINECO, Spain., Peer reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Learning general policies from small examples without supervision

  • Francès, Guillem
  • Bonet, Blai
  • Geffner, Héctor
Generalized planning is concerned with the computation of general policies that solve multiple instances of a planning domain all at once. It has been recently shown that these policies can be computed in two steps: first, a suitable abstraction in the form of a qualitative numerical planning problem (QNP) is learned from sample plans, then the general policies are obtained from the learned QNP using a planner. In this work, we introduce an alternative approach for computing more expressive general policies which does not require sample plans or a QNP planner. The new formulation is very simple and can be cast in terms that are more standard in machine learning: a large but finite pool of features is defined from the predicates in the planning examples using a general grammar, and a small subset of features is sought for separating “good” from “bad” state transitions, and goals from non-goals. The problems of finding such a “separating surface” while labeling the transitions as “good” or “bad” are jointly addressed as a single combinatorial optimization problem expressed as a Weighted Max-SAT problem. The advantage of looking for the simplest policy in the given feature space that solves the given examples, possibly non-optimally, is that many domains have no general, compact policies that are optimal. The approach yields general policies for a number of benchmark domains., This research is partially funded by an ERC Advanced Grant (No 885107), by grant TIN-2015-67959-P from MINECO, Spain, and by the Knut and Alice Wallenberg (KAW) Foundation through the WASP program. H. Geffner is also a Wallenberg Guest Professor at Linköping University, Sweden. G. Francès is partially supported by grant IJC2019-039276-I from MICINN, Spain.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Deep reinforcement learning in medicine

  • Jonsson, Anders, 1973-
Reinforcement learning has achieved tremendous success in recent years, notably in complex games such as Atari, Go, and chess. In large part, this success has been made possible by powerful function approximation methods in the form of deep neural networks. The objective of this paper is to introduce the basic concepts of reinforcement learning, explain how reinforcement learning can be effectively combined with deep learning, and explore how deep reinforcement learning could be useful in a medical context., This work is partially funded by the grant TIN2015-67959 of the Spanish Ministry of Science.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Purely declarative action descriptions are overrated: classical planning with simulators

  • Francès, Guillem
  • Ramírez Jávega, Miquel
  • Lipovetzky, Nir
  • Geffner, Héctor
Comunicació presentada a la Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI 2017), celebrada els dies 19 a 25 d'agost a Melbourne, Austràlia., Classical planning is concerned with problems where a goal needs to be reached from a known initial state by doing actions with deterministic, known effects. Classical planners, however, deal only with classical problems that can be expressed in declarative planning languages such as STRIPS or PDDL. This prevents their use on problems that are not easy to model declaratively or whose dynamics are given via simulations. Simulators do not provide a declarative representation of actions, but simply return successor states. The question we address in this paper is: can a planner that has access to the structure of states and goals only, approach the performance of planners that also have access to the structure of actions expressed in PDDL? To answer this, we develop domain-independent, black box planning algorithms that completely ignore action structure, and show that they match the performance of state-of-the-art classical planners on the standard planning benchmarks. Effective black box algorithms open up new possibilities for modeling and for expressing control knowledge, which we also illustrate., G. Francès is supported by the M. de Maeztu Programme (MDM-2015-0502) and H. Geffner by grant TIN2015-67959-P, both from MINECO, Spain. M. Ramirez and N. Lipovetzky have been partially funded by the Australian DST Group.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Generating context-free grammars using classical planning

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders, 1973-
Comunicació presentada a: IJCAI International Joint Conference on Artificial Intelligence, celebrada a Melbourne, Austràlia, del 19 al 25 d'agost de 2017., This paper presents a novel approach for generating Context-Free Grammars (CFGs) from small sets of input strings (a single input string in some cases). Our approach is to compile this task into a classical planning problem whose solutions are sequences of actions that build and validate a CFG compliant with the input strings. In addition, we show that our compilation is suitable for implementing the two canonical tasks for CFGs, string production and string recognition., This work is partially supported by grant TIN2015-67959 and the Maria de Maeztu Units of Excellence Programme MDM-2015-0502, MEC, Spain.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

∃-STRIPS: existential quantification in planning and constraint satisfaction

  • Francès, Guillem
  • Geffner, Héctor
Comunicació presentada a la Twenty-Fifth International Joint Conference on Artificial Intelligence, celebrada els dies 9 a 15 de juliol de 2016 a Nova York., Existentially quantified variables in goals and action preconditions are part of the standard PDDL planning language, yet few planners support them, while those that do compile them away at an exponential cost. In this work, we argue that existential variables are an essential feature for representing and reasoning with constraints in planning, and that it is harmful to compile them away or avoid them altogether, since this hides part of the problem structure that can be exploited computationally. We show how to do this by formulating an extension of the standard delete-relaxation heuristics that handles existential variables. While this extension is simple, the consequences for both modeling and computation are important. Furthermore, by allowing existential variables in STRIPS and treating them properly, CSPs can be represented and solved in a direct manner as action-less,fluent-less STRIPS planning problems, something important for problems involving restrictions. In addition, functional fluents in Functional STRIPS can be compiled away with no effect on the structure and informativeness of the resulting heuristic. Experiments are reported comparing our native ∃-STRIPS planner with state-of-the-art STRIPS planners over compiled and propositional encodings, and with a Functional STRIPS planner., This work is partially supported by the M. de Maeztu Programme (MDM-2015-0502) and by grants TIN2015-67959 and CSD2010-00034, MEC, Spain.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Generalized planning as heuristic search

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders, 1973-
Comunicació presentada a ICAPS2021, celebrat del 2 a 13 d'agost de 2021 de manera virtual., Although heuristic search is one of the most successful approaches to classical planning, this planning paradigm does not apply straightforwardly to Generalized Planning (GP). Planning as heuristic search traditionally addresses the computation of sequential plans by searching in a grounded state-space. On the other hand, GP aims at computing algorithm-like plans that can branch and loop, and that generalize to a (possibly infinite) set of planning instances. This paper adapts the {\em planning as heuristic search} paradigm to the particularities of GP and presents the first native heuristic search approach to GP. First, the paper defines a novel GP solution space that is independent of the number of planning instances in a GP problem, and the size of these instances. Second, the paper defines several evaluation and heuristic functions, that do not require grounding, for guiding a combinatorial search in the GP solution space, making it possible to handle state variables with large numerical domains (e.g. integers). Lastly, the paper defines an algorithm for GP called Best-First Generalized Planning (BFGP), that implements a best-first search in the solution space guided by our evaluation/heuristic functions., Javier Segovia-Aguas is supported by TAILOR, a project funded by EU H2020 research and innovation programme no. 952215, an ERC Advanced Grant no. 885107, and grant TIN-2015-67959-P from MINECO, Spain. Sergio Jiménez is supported by the Ramon y Cajal program, RYC-2015-18009, the Spanish MINECO project TIN2017-88476-C2-1-R. Anders Jonsson is partially supported by Spanish grants PID2019-108141GB-I00 and PCIN-2017-082.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

General policies, representations, and planning width

  • Bonet, Blai
  • Geffner, Héctor
It has been observed that in many of the benchmark planning domains, atomic goals can be reached with a simple polynomial exploration procedure, called IW, that runs in time exponential in the problem width. Such problems have indeed a bounded width: a width that does not grow with the number of problem variables and is often no greater than two. Yet, while the notion of width has become part of the state- of-the-art planning algorithms like BFWS, there is still no good explanation for why so many benchmark domains have bounded width. In this work, we address this question by relating bounded width and serialized width to ideas of generalized planning, where general policies aim to solve multiple instances of a planning problem all at once. We show that bounded width is a property of planning domains that admit optimal general policies in terms of features that are explicitly or implicitly represented in the domain encoding. The results are extended to the larger class of domains with bounded serialized width where the general policies do not have to be optimal. The study leads also to a new simple, meaningful, and expressive language for specifying domain serializations in the form of policy sketches which can be used for encoding domain control knowledge by hand or for learning it from traces. The use of sketches and the meaning of the theoretical results are all illustrated through a number of examples., The research is partially funded by an ERC Advanced Grant (No 885107), by grant TIN-2015-67959-P from MINECO, Spain, and by the Knut and Alice Wallenberg (KAW) Foundation through the WASP program. H. Geffner is a also Wallenberg Guest Professor at Linkoping University, Sweden.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Computing hierarchical finite state controllers with classical planning

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders, 1973-
Finite State Controllers (FSCs) are an effective way to compactly represent sequential plans. By imposing appropriate conditions on transitions, FSCs can also represent generalized plans (plans that solve a range of planning problems from a given domain). In this paper we introduce the concept of hierarchical FSCs for planning by allowing controllers to call other controllers. This call mechanism allows hierarchical FSCs to represent generalized plans more compactly than individual FSCs, to compute controllers in a modular fashion or even more, to compute recursive controllers. The paper introduces a classical planning compilation for computing hierarchical FSCs that solve challenging generalized planning tasks. The compilation takes as input a finite set of classical planning problems from a given domain. The output of the compilation is a single classical planning problem whose solution induces: (1) a hierarchical FSC and (2), the corresponding validation of that controller on the input classical planning problems., This work is partially supported by grant TIN2015-67959 and the Maria de Maeztu Units of Excellence Programme MDM-2015-0502, MEC, Spain. Sergio Jiménez is supported by the RYC15/18009, program funded by the Spanish government.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Constructing hierarchical task models using invariance analysis

  • Jonsson, Anders, 1973-
  • Lotinac, Damir
Comunicació presentada a la biennial European Conference on Artificial Intelligence (ECAI 2016), 22nd European Conference on Artificial Intelligence, celebrada a La Haia (Països Baixos) del 29 d'agost al 2 de septembre de 2016., Hierarchical Task Networks (HTNs) are a common model for encoding knowledge about planning domains in the form of task decompositions. We present a novel algorithm that uses invariant analysis to construct an HTN from the PDDL description of a planning domain and a single representative instance. The algorithm defines two types of composite tasks that interact to achieve the goal of a planning instance. One type of task achieves fluents by traversing invariants in which only one fluent can be true at a time. The other type of task applies a single action, which first involves ensuring that the precondition of the action holds. The resulting HTN can be applied to any instance of the planning domain, and is provably sound. We show that the performance of our algorithm is comparable to algorithms that learn HTNs from examples and use added knowledge., This work is partially supported by the MINECO/FEDER grant TIN2015-67959 and the Maria de Maeztu Units of Excellence Programme MDM-2015-0502, MEC, Spain.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Computing programs for generalized planning using a classical planner

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders
Although heuristic search is one of the most successful approaches to classical planning, this planning paradigm does not apply straightforwardly to Generalized Planning (GP). This paper adapts the planning as heuristic search paradigm to the particularities of GP, and presents the first native heuristic search approach to GP. First, the paper defines a program-based solution space for GP that is independent of the number of planning instances in a GP problem, and the size of these instances. Second, the paper defines the BFGP algorithm for GP, that implements a best-first search in our programbased solution space, and that is guided by different evaluation and heuristic functions.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Solving multiagent planning problems with concurrent conditional effects

  • Furelos Blanco, Daniel
  • Jonsson, Anders, 1973-
Comunicació presentada al 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019 and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, celebrat del 27 de gener a l'1 de febrer de 2019 a Palo Alta, EEUU., In this work we present a novel approach to solving concurrent multiagent planning problems in which several agents act in parallel. Our approach relies on a compilation from concurrent multiagent planning to classical planning, allowing us to use an off-the-shelf classical planner to solve the original multiagent problem. The solution can be directly interpreted as a concurrent plan that satisfies a given set of concurrency constraints, while avoiding the exponential blowup associated with concurrent actions. Our planner is the first to handle action effects that are conditional on what other agents are doing. Theoretically, we show that the compilation is sound and complete. Empirically, we show that our compilation can solve challenging multiagent planning problems that require concurrent actions., This work has been supported by the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502). Anders Jonsson is partially supported by the grants TIN2015-67959 and PCIN-2017-082 of the Spanish Ministry of Science.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Hierarchical finite state controllers for generalized planning

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders, 1973-
Finite State Controllers (FSCs) are an effective way/nto represent sequential plans compactly. By imposing/nappropriate conditions on transitions, FSCs/ncan also represent generalized plans that solve a/nrange of planning problems from a given domain./nIn this paper we introduce the concept of hierarchical/nFSCs for planning by allowing controllers/nto call other controllers. We show that hierarchical/nFSCs can represent generalized plans more compactly/nthan individual FSCs. Moreover, our call/nmechanism makes it possible to generate hierarchical/nFSCs in a modular fashion, or even to apply/nrecursion. We also introduce a compilation that/nenables a classical planner to generate hierarchical/nFSCs that solve challenging generalized planning/nproblems. The compilation takes as input a set of/nplanning problems from a given domain and outputs/na single classical planning problem, whose solution/ncorresponds to a hierarchical FSC., This work is partially supported by grant TIN2015-67959 and the/nMaria de Maeztu Units of Excellence Programme MDM-2015-/n0502, MEC, Spain. Sergio Jimenez is partially supported by the /nJuan de la Cierva program funded by the Spanish government.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Lifelong control of off-grid microgrid with model-based reinforcement learning

  • Totaro, Simone
  • Boukas, Ioannis
  • Jonsson, Anders, 1973-
  • Cornélusse, Bertrand
Off-grid microgrids are receiving a growing interest for rural electrification purposes in developing countries due to their ability to ensure affordable, sustainable and reliable energy services. Off-grid microgrids rely on renewable energy sources (RES) coupled with storage systems to supply the electrical consumption. The inherent uncertainty introduced by RES as well as the stochastic nature of the electrical demand in rural contexts pose significant challenges to the efficient control of off-grid microgrids throughout their entire life span. In this paper, we address the lifelong control problem of an isolated microgrid. We categorize the set of changes that may occur over its life span in progressive and abrupt changes. We propose a novel model-based reinforcement learning algorithm that is able to address both types of changes. In particular, the proposed algorithm demonstrates generalisation properties, transfer capabilities and better robustness in case of fast-changing system dynamics. The proposed algorithm is compared against a rule-based policy and a model predictive controller with look-ahead. The results show that the trained agent is able to outperform both benchmarks in the lifelong setting where the system dynamics are changing over time., This research is carried out in the framework of the project Dynamically Evolving Long-Term Autonomy (DELTA), a European research project funded under the CHIST-ERA scheme (http://www.chistera.eu/). Anders Jonsson is partially supported by the grants TIN2015-67959 and PCIN-2017-082 of the Spanish Ministry of Science. The authors would like to thank Sergio Balderrama for the provision of measured data from the “El Espino” microgrid in Bolivia and Alessandro Davide Ialongo for the fruitful discussion.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Automatic generation of high-level state features for generalized planning

  • Lotinac, Damir
  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders, 1973-
In many domains generalized plans can only/nbe computed if certain high-level state features,/ni.e. features that capture key concepts to accurately/ndistinguish between states and make good decisions,/nare available. In most applications of generalized/nplanning such features are hand-coded by/nan expert. This paper presents a novel method/nto automatically generate high-level state features/nfor solving a generalized planning problem. Our/nmethod extends a compilation of generalized planning/ninto classical planning and integrates the computation/nof generalized plans with the computation/nof features, in the form of conjunctive queries. Experiments/nshow that we generate features for diverse/ngeneralized planning problems and hence,/ncompute generalized plans without providing a/nprior high-level representation of the states. We/nalso bring a new landscape of challenging benchmarks/nto classical planning since our compilation/nnaturally models classification tasks as classical/nplanning problems., This work is partially supported by grant TIN2015-67959 and the/nMaria de Maeztu Units of Excellence Programme MDM-2015-/n0502, MEC, Spain. Sergio Jimenez is partially supported by the /nJuan de la Cierva program funded by the Spanish government.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Unsupervised classification of planning instances

  • Segovia-Aguas, Javier
  • Jiménez, Sergio
  • Jonsson, Anders, 1973-
Comunicació presentada a: The 27th International Conference on Automated Planning and Scheduling, ICAPS 2017, celebrada a Pittsburgh, Estats Units, del 18 al 23 de juny de 2017, In this paper we introduce a novel approach for unsupervised classification of planning instances based on the recent formalism of planning programs. Our approach is inspired by structured prediction in machine learning, which aims at predicting structured information about a given input rather than a scalar value. In our case, each input is an unlabelled classical planning instance, and the associated structured information is the planning program that solves the instance. We describe a method that takes as input a set of planning instances and outputs a set of planning programs, classifying each instance according to the program that solves it. Our results show that automated planning can be successfully used to solve structured unsupervised classification tasks, and invites further exploration of the connection between automated planning and structured prediction., This work is partially supported by grant TIN2015-67959 and the Maria de Maeztu Units of Excellence Programme MDM-2015-0502, MEC, Spain.