DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication
Artículo científico (JournalArticle)
A Virtual Reality Framework For Fast Dataset Creation Applied to Cloth Manipulation with Automatic Semantic Labelling
Trabajo presentado en la IEEE International Conference on Robotics and Automation (ICRA), celebrada en Londres (UK), del 29 de mayo al 2 de junio de 2024, Teaching complex manipulation skills, such as folding garments, to a bi-manual robot is a very challenging task, which is often tackled through learning from demon-stration. The few datasets of garment-folding demonstrations available nowadays to the robotics research community have been either gathered from human demonstrations or generated through simulation. The former have the great difficulty of perceiving both cloth state and human action as well as transferring them to the dynamic control of the robot, while the latter require coding human motion into the simulator in open loop, i.e., without incorporating the visual feedback naturally used by people, resulting in far-from-realistic movements. In this article, we present an accurate dataset of human cloth folding demonstrations. The dataset is collected through our novel virtual reality (VR) framework, based on Unity's 3D platform and the use of an HTC Vive Pro system. The framework is capable of simulating realistic garments while allowing users to interact with them in real time through handheld controllers. By doing so, and thanks to the immersive experience, our framework permits exploiting human visual feedback in the demonstrations while at the same time getting rid of the difficulties of capturing the state of cloth, thus simplifying data acquisition and resulting in more realistic demonstrations. We create and make public a dataset of cloth manipulation sequences, whose cloth states are semantically labeled in an automatic way by using a novel low-dimensional cloth representation that yields a very good separation between different cloth configurations., The research leading to these results receives funding from the European
Research Council (ERC) from the European Union Horizon 2020 Programme
under grant agreement no. 741930 (CLOTHILDE: CLOTH manIpulation
Learning from DEmonstrations) and project SoftEnable (HORIZONCL4-
2021-DIGITAL-EMERGING-01-101070600). Authors also received
funding from project CHLOE-GRAPH (PID2020-118649RB-I00) funded by
MCIN/ AEI /10.13039/501100011033 and COHERENT (PCI2020-120718-
2) funded by MCIN/ AEI /10.13039/501100011033 and cofunded by the
”European Union NextGenerationEU/PRTR”., Peer reviewed
Project:
EC / H2020 741930
,
AEI / Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PID2020-118649RB-I00
,
AEI / Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PCI2020-120718-2
,
EC / HE 101070600
DOI: http://hdl.handle.net/10261/354962, https://api.elsevier.com/content/abstract/scopus_id/85162172346
HANDLE: http://hdl.handle.net/10261/354962, https://api.elsevier.com/content/abstract/scopus_id/85162172346