Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 13
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Dual-arm robotic manipulation using visual guidance

  • Ramon Canyameres, Pol|||0009-0005-5548-7527
  • Palomo Avellaneda, Leopold|||0000-0002-8734-1566
  • Zaplana Agut, Isiah|||0000-0002-0862-3240
  • Rosell Gratacòs, Jan|||0000-0003-4854-2370
Dual-arm manipulation poses many challenges in order to actually perform manipulation tasks with enough efficiency. To cope with this challenges, this work-in-progress describes a system for motion coordination of a mobile anthropomorphic dual-arm robot that focuses on the capability for performing manipulations actions with precision by means of a visual guidance provided by a camera mounted on an articulated head. The proposal presents different control modes, evaluated with real experiments, and discusses the main future research lines., This work was supported by the European Commission’s Horizon Europe Framework Programme with the project IntelliMan (AI-Powered Manipulation System for Advanced Robotic Service, Manufacturing and Prosthetics) under Grant Agreement 101070136., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Knowledge-based execution configuration for adaptive behavior trees

  • Ruiz Celada, Oriol|||0000-0002-6171-7270
  • Rosell Gratacòs, Jan|||0000-0003-4854-2370
  • Suárez Feijóo, Raúl|||0000-0002-3853-7095
Automated planning is commonly used to obtain plans to solve particular tasks. To execute these plans, Behavior Trees have emerged as a popular execution architecture due to their reactivity and modularity. Configuring the execution of a plan into a Behavior Tree requires expanding the high level actions into the proper Behavior Tree structure. Typically, this is achieved using of pre-defined rigid templates. In this work, we propose a novel approach to generating the Behavior Trees using ontologies. The generated Behavior Trees are tailored to the specific requirements of a task and the world by using modifiers to a base template that provides a general solution to the task. These modifiers and their properties are formally defined using ontologies. A proof of concept is developed, illustrating how the Behavior Trees for the execution of manipulation tasks in a kitchen scenario can be adapted to varying conditions by applying different modifiers., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work was supported by the European Commission’s Horizon Europe Framework Programme with the project AI-Powered Manipulation System for Advanced Robotic Service, Manufacturing and Prosthetics (IntelliMan) under Grant Agreement 101070136., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Estudio, informe, memoria (Report)

Knowledge-based execution configuration of behavior trees

  • Ruiz Celada, Oriol|||0000-0002-6171-7270
  • Rosell Gratacòs, Jan|||0000-0003-4854-2370
  • Suárez Feijóo, Raúl|||0000-0002-3853-7095
3rd edition of the Workshop on Ontologies and Standards for Robotics and Automation, Automated planning is commonly used to obtain plans to solve particular tasks. To execute the plans, Behavior Trees (BTs) have emerged as a popular execution architecture due to their reactivity and modularity. Configuring the execution of a plan into a BT requires expanding high level actions into the proper BT structure. For this, the use of pre-defined rigid templates is common. In this work, in order to overcome the limitation of these methods, we present a new approach to describe robotic behaviors using ontologies and to adapt BTs to current specifications of the task and the world by reasoning on what modifiers to their base template need to be applied. These modifiers and their properties are formally defined using ontologies. We present a proof of concept of these modifiers applied to a base BT of a Pick operation., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Smart perception for situation awareness in robotic manipulation tasks

  • Ruiz Celada, Oriol|||0000-0002-6171-7270
  • Dalmases, Albert
  • Zaplana Agut, Isiah|||0000-0002-0862-3240
  • Rosell Gratacòs, Jan|||0000-0003-4854-2370
Robotic manipulation in semi-structured environments require perception, planning and execution capabilities to be robust to deviations and adaptive to changes, and knowledge representation and reasoning may play a role in this direction in order to make robots aware of the situations, of the planning domains and of their own execution structures. This paper proposes an approach aimed at enhancing the perception capabilities of robotic systems through the integration of various technologies. In particular, the novelties of the proposed smart perception module include the combination of visual sensor data, object detection, and pose estimation techniques, leveraging a fiducial markers and deep learning-based methods, being able to integrate multiple sensors and perception pipelines. In addition, reasoning capabilities are introduced through the utilization of ontologies. The result is a robust and smart perception system capable of handling both simulated and real-world scenarios which in turn provides the required functionalities to allow the robot to understand its surroundings, with a primary focus on robotic manipulation tasks. The discussion on the tools used and the key implementation details are included, as well as the results in some simulated and real scenarios that validate the proposal as a module that provides situation awareness to allow a manipulation framework to adapt the robot actions to uncertain and changing scenarios., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Onto-LLM-TAMP: knowledge-oriented task and motion planning using large language models

  • Ud Din, Muhayy
  • Rosell Gratacòs, Jan|||0000-0003-4854-2370
  • Akram, Waseem|||0000-0002-7401-5120
  • Zaplana Agut, Isiah|||0000-0002-0862-3240
  • Roa Garzón, Máximo Alejandro
  • Hussain, Irfan|||0000-0003-2759-0306
Performing complex manipulation tasks in dynamic environments requires efficient Task and Motion Planning (TAMP) approaches that combine high-level symbolic plans with low-level motion control. Advances in Large Language Models (LLMs), such as GPT-4, are transforming task planning by offering natural language as an intuitive and flexible way to describe tasks, generate symbolic plans, and reason. However, the effectiveness of LLM-based TAMP approaches is limited due to static and template-based prompting, which limits adaptability to dynamic environments and complex task contexts. To address these limitations, this work proposes a novel Onto-LLM-TAMP framework that employs knowledge-based reasoning to refine and expand user prompts with task-contextual reasoning and knowledge-based environment state descriptions. Integrating domain-specific knowledge into the prompt ensures semantically accurate and context-aware task plans. The proposed framework demonstrates its effectiveness by resolving semantic errors in symbolic plan generation, such as maintaining logical temporal goal ordering in scenarios involving hierarchical object placement. The proposed framework is validated through both simulation and real-world scenarios, demonstrating significant improvements over the baseline approach in terms of adaptability to dynamic environments and the generation of semantically correct task plans., This work has been supported by the Khalifa University of Science and Technology under Award No. RC1-2018-KUCARS, Silal Innovation Oasis through the projects under grants 8475000023, and by the European Commission’s Horizon Europe Framework Programme with the project IntelliMan (AI-Powered Manipulation System for Advanced Robotic Service, Manufacturing and Prosthetics) under Grant Agreement 101070136.
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Using ontologies for adaptive planning and robust execution in robotic manipulation

  • Ruiz, Oriol
  • Molina, Víctor
  • Suárez Feijóo, Raúl|||0000-0002-3853-7095
  • Ud Din, Muhayy
  • Zaplana Agut, Isiah|||0000-0002-0862-3240
  • Rosell Gratacòs, Jan|||0000-0003-4854-2370
Robotic manipulation tasks face many challenges when dealing with real scenarios that require the perception and understanding of the environment, planning with procedures that can adapt to the actual situation, and monitoring and recovery capabilities to ensure successful execution despite uncertainties and errors. The use of knowledge in terms of ontologies can address these needs by providing reasoning capabilities that allow the robot to be aware of the situation and the task and to dynamically create execution structures that adapt its behaviors in a flexible and robust manner. In this line, this paper discusses the work done at the Institute of Industrial and Control Engineering (IOC-UPC) that uses ontologies combined with Large Language Models to plan and automatically generate Behavior Trees with monitoring and recovery capabilities, which allow for the control of robot execution for successful performance in real scenarios., This work was partially supported by the European Commission’s Horizon Europe Framework Programme with the project IntelliMan (AI-Powered Manipulation System for Advanced Robotic Service, Manufacturing, and Prosthetics) under Grant Agreement 101070136., Peer Reviewed
Zaguán. Repositorio Digital de la Universidad de Zaragoza
Publicación Artículo científico (JournalArticle)

Bayesian optimization for robust robotic grasping using a sensorized compliant hand

  • Lechuz-Sierra, Juan G.
  • Martín, Ana Elvira H.
  • Sundaram, Ashok M.
  • Martinez-Cantín, Rubén
  • Roa, Máximo A.
One of the first tasks we learn as children is to grasp objects based on our tactile perception. Incorporating such skill in robots will enable multiple applications, such as increasing flexibility in industrial processes or providing assistance to people with physical disabilities. However, the difficulty lies in adapting the grasping strategies to a large variety of tasks and objects, which can often be unknown. The brute-force solution is to learn new grasps by trial and error, which is inefficient and ineffective. In contrast, Bayesian optimization applies active learning by adding information to the approximation of an optimal grasp. This paper proposes the use of Bayesian optimization techniques to safely perform robotic grasping. We analyze different grasp metrics to provide realistic grasp optimization in a real system including tactile sensors. An experimental evaluation in the robotic system shows the usefulness of the method for performing unknown object grasping even in the presence of noise and uncertainty inherent to a real-world environment.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Knowledge-based execution configuration for adaptive behavior trees

  • Ruiz Celada, Oriol
  • Rosell Gratacòs, Jan
  • Suárez Feijóo, Raúl
Automated planning is commonly used to obtain plans to solve particular tasks. To execute these plans, Behavior Trees have emerged as a popular execution architecture due to their reactivity and modularity. Configuring the execution of a plan into a Behavior Tree requires expanding the high level actions into the proper Behavior Tree structure. Typically, this is achieved using of pre-defined rigid templates. In this work, we propose a novel approach to generating the Behavior Trees using ontologies. The generated Behavior Trees are tailored to the specific requirements of a task and the world by using modifiers to a base template that provides a general solution to the task. These modifiers and their properties are formally defined using ontologies. A proof of concept is developed, illustrating how the Behavior Trees for the execution of manipulation tasks in a kitchen scenario can be adapted to varying conditions by applying different modifiers., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work was supported by the European Commission’s Horizon Europe Framework Programme with the project AI-Powered Manipulation System for Advanced Robotic Service, Manufacturing and Prosthetics (IntelliMan) under Grant Agreement 101070136., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Onto-LLM-TAMP: knowledge-oriented task and motion planning using large language models

  • Ud Din, Muhayy
  • Rosell Gratacòs, Jan
  • Akram, Waseem
  • Zaplana Agut, Isiah
  • Roa Garzón, Máximo Alejandro
  • Hussain, Irfan
Performing complex manipulation tasks in dynamic environments requires efficient Task and Motion Planning (TAMP) approaches that combine high-level symbolic plans with low-level motion control. Advances in Large Language Models (LLMs), such as GPT-4, are transforming task planning by offering natural language as an intuitive and flexible way to describe tasks, generate symbolic plans, and reason. However, the effectiveness of LLM-based TAMP approaches is limited due to static and template-based prompting, which limits adaptability to dynamic environments and complex task contexts. To address these limitations, this work proposes a novel Onto-LLM-TAMP framework that employs knowledge-based reasoning to refine and expand user prompts with task-contextual reasoning and knowledge-based environment state descriptions. Integrating domain-specific knowledge into the prompt ensures semantically accurate and context-aware task plans. The proposed framework demonstrates its effectiveness by resolving semantic errors in symbolic plan generation, such as maintaining logical temporal goal ordering in scenarios involving hierarchical object placement. The proposed framework is validated through both simulation and real-world scenarios, demonstrating significant improvements over the baseline approach in terms of adaptability to dynamic environments and the generation of semantically correct task plans., This work has been supported by the Khalifa University of Science and Technology under Award No. RC1-2018-KUCARS, Silal Innovation Oasis through the projects under grants 8475000023, and by the European Commission’s Horizon Europe Framework Programme with the project IntelliMan (AI-Powered Manipulation System for Advanced Robotic Service, Manufacturing and Prosthetics) under Grant Agreement 101070136., Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación

Knowledge-based execution configuration of behavior trees

  • Ruiz Celada, Oriol
  • Rosell Gratacòs, Jan
  • Suárez Feijóo, Raúl
3rd edition of the Workshop on Ontologies and Standards for Robotics and Automation, Automated planning is commonly used to obtain plans to solve particular tasks. To execute the plans, Behavior Trees (BTs) have emerged as a popular execution architecture due to their reactivity and modularity. Configuring the execution of a plan into a BT requires expanding high level actions into the proper BT structure. For this, the use of pre-defined rigid templates is common. In this work, in order to overcome the limitation of these methods, we present a new approach to describe robotic behaviors using ontologies and to adapt BTs to current specifications of the task and the world by reasoning on what modifiers to their base template need to be applied. These modifiers and their properties are formally defined using ontologies. We present a proof of concept of these modifiers applied to a base BT of a Pick operation., Peer Reviewed, Preprint
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Using ontologies for adaptive planning and robust execution in robotic manipulation

  • Ruiz, Oriol
  • Molina, Víctor
  • Suárez Feijóo, Raúl
  • Ud Din, Muhayy
  • Zaplana Agut, Isiah
  • Rosell Gratacòs, Jan
Robotic manipulation tasks face many challenges when dealing with real scenarios that require the perception and understanding of the environment, planning with procedures that can adapt to the actual situation, and monitoring and recovery capabilities to ensure successful execution despite uncertainties and errors. The use of knowledge in terms of ontologies can address these needs by providing reasoning capabilities that allow the robot to be aware of the situation and the task and to dynamically create execution structures that adapt its behaviors in a flexible and robust manner. In this line, this paper discusses the work done at the Institute of Industrial and Control Engineering (IOC-UPC) that uses ontologies combined with Large Language Models to plan and automatically generate Behavior Trees with monitoring and recovery capabilities, which allow for the control of robot execution for successful performance in real scenarios., This work was partially supported by the European Commission’s Horizon Europe Framework Programme with the project IntelliMan (AI-Powered Manipulation System for Advanced Robotic Service, Manufacturing, and Prosthetics) under Grant Agreement 101070136., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Dual-arm robotic manipulation using visual guidance

  • Ramon Canyameres, Pol
  • Palomo Avellaneda, Leopold
  • Zaplana Agut, Isiah
  • Rosell Gratacòs, Jan
Dual-arm manipulation poses many challenges in order to actually perform manipulation tasks with enough efficiency. To cope with this challenges, this work-in-progress describes a system for motion coordination of a mobile anthropomorphic dual-arm robot that focuses on the capability for performing manipulations actions with precision by means of a visual guidance provided by a camera mounted on an articulated head. The proposal presents different control modes, evaluated with real experiments, and discusses the main future research lines., This work was supported by the European Commission’s Horizon Europe Framework Programme with the project IntelliMan (AI-Powered Manipulation System for Advanced Robotic Service, Manufacturing and Prosthetics) under Grant Agreement 101070136., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Smart perception for situation awareness in robotic manipulation tasks

  • Ruiz Celada, Oriol
  • Dalmases, Albert
  • Zaplana Agut, Isiah
  • Rosell Gratacòs, Jan
Robotic manipulation in semi-structured environments require perception, planning and execution capabilities to be robust to deviations and adaptive to changes, and knowledge representation and reasoning may play a role in this direction in order to make robots aware of the situations, of the planning domains and of their own execution structures. This paper proposes an approach aimed at enhancing the perception capabilities of robotic systems through the integration of various technologies. In particular, the novelties of the proposed smart perception module include the combination of visual sensor data, object detection, and pose estimation techniques, leveraging a fiducial markers and deep learning-based methods, being able to integrate multiple sensors and perception pipelines. In addition, reasoning capabilities are introduced through the utilization of ontologies. The result is a robust and smart perception system capable of handling both simulated and real-world scenarios which in turn provides the required functionalities to allow the robot to understand its surroundings, with a primary focus on robotic manipulation tasks. The discussion on the tools used and the key implementation details are included, as well as the results in some simulated and real scenarios that validate the proposal as a module that provides situation awareness to allow a manipulation framework to adapt the robot actions to uncertain and changing scenarios., Peer Reviewed, Postprint (published version)