Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 18
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Exploring the role of expected collision feedback in crowded virtual environments

  • Yun, Haoran
  • Pontón Martínez, José Luis|||0000-0001-6576-4528
  • Beacco Porres, Alejandro|||0000-0001-8192-1431
  • Andújar Gran, Carlos Antonio|||0000-0002-8480-4713
  • Pelechano Gómez, Núria|||0000-0002-1437-245X
An increasing number of virtual reality applications require environments that emulate real-world conditions. These environments often involve dynamic virtual humans showing realistic behaviors. Understanding user perception and navigation among these virtual agents is key for designing realistic and effective environments featuring groups of virtual humans. While collision risk significantly influences human locomotion in the real world, this risk is largely absent in virtual settings. This paper studies the impact of the expected collision feedback on user perception and interaction with virtual crowds. We examine the effectiveness of commonly used collision feedback techniques (auditory cues and tactile vibrations) as well as inducing participants to expect that a physical bump with a real person might occur, as if some virtual humans actually correspond to real persons embodied into them and sharing the same physical space. Our results indicate that the expected collision feedback significantly influences both participant behavior—encompassing global navigation and local movements—and subjective perceptions of presence and copresence. Specifically, the introduction of a perceived risk of actual collision was found to significantly impact global navigation strategies and increase the sense of presence. Auditory cues had a similar effect on global navigation and additionally enhanced the sense of copresence. In contrast, vibrotactile feedback was primarily effective in influencing local movements., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860768 (CLIPE project), and from MCIN/AEI/10.13039/501100011033/FEDER, UE (PID2021- 122136OB-C21), ”A way to make Europe”. Jose Luis Ponton was also funded by the Spanish Ministry of Universities (FPU21/01927)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Authoring virtual crowds: a survey

  • Lemonari, Marilena
  • Blanco Guerra, Rafael Andrés
  • Charalambous, Panayiotis
  • Pelechano Gómez, Núria|||0000-0002-1437-245X
  • Avraamides, Marios
  • Pettré, Julien
  • Chrysanthou, Yiorgos
Recent advancements in crowd simulation unravel a wide range of functionalities for virtual agents, delivering highly-realistic,natural virtual crowds. Such systems are of particular importance to a variety of applications in fields such as: entertainment(e.g., movies, computer games); architectural and urban planning; and simulations for sports and training. However, providingtheir capabilities to untrained users necessitates the development of authoring frameworks. Authoring virtual crowds is acomplex and multi-level task, varying from assuming control and assisting users to realise their creative intents, to deliveringintuitive and easy to use interfaces, facilitating such control. In this paper, we present a categorisation of the authorable crowdsimulation components, ranging from high-level behaviours and path-planning to local movements, as well as animation andvisualisation. We provide a review of the most relevant methods in each area, emphasising the amount and nature of influencethat the users have over the final result. Moreover, we discuss the currently available authoring tools (e.g., graphical userinterfaces, drag-and-drop), identifying the trends of early and recent work. Finally, we suggest promising directions for futureresearch that mainly stem from the rise of learning-based methods, and the need for a unified authoring framework., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No 860768 (CLIPE project). This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy, Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Social crowd simulation: improving realism with social rules and gaze behavior

  • Itatani, Reiya|||0000-0002-8430-1657
  • Pelechano Gómez, Núria|||0000-0002-1437-245X
Current crowd simulation models focus mostly on steering towards a goal while avoiding collisions based on the agent’s direction of movement. This leads to robot-like simulations since agents’ appear to always have their attention perfectly aligned with the direction of movement. In the real world, we observe that humans move in a crowd performing collision avoidance driven by attention, gaze, and non-verbal coordination with incoming traffic. In addition, humans exhibit different steering strategies based on whether they walk alone or in a group, whether they can look ahead and plan their best local movement, or react more abruptly because their gaze diverts from their direction of movement. Human gaze can be driven by movement, but also by distractions such as being engaged in conversation with other people or using mobile phones. These human features are overlooked in crowd simulation, often leading to perfectly smooth local movements of individuals. Unfortunately, this lack of social behaviour and variety in animations may be perceived as unrealistic when observing the results on a 2D display, and it may become even more apparent in immersive scenarios where the participant is at eye level with the virtual humans. This paper proposes a novel approach to enhance the realism of a rule-based crowd simulation model by incorporating social rules and gaze-driven attention with consistent animations. The ultimate goal is to make immersive virtual crowds more realistic. Our proposed method enhances existing crowd simulation frameworks by integrating social behavior models that affect both individual and collective dynamics, and drives gaze behavior to better simulate attention. We conducted validation user studies on both a 2D display and in immersive VR, and observed that applying these models to both the steering and animation levels significantly improves the realism of the crowd simulation. The 2D display based user study based on video comparisons showed that our model was perceived as more realistic and consistent with social behaviors compared to traditional collision avoidance approaches, that used only locomotion or random animations. The immersive user study showed that participants effectively detected the social behaviours included in our model as intended. The results revealed significant differences in the participants’ perceptions of the various behaviours exhibited by our social crowd model., This work was supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 860768 (CLIPE project), and by MCIN/AEI/10.13039/501100011033/FEDER, UE (PID2021-122136OBC21)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

SparsePoser: Real-time full-body motion reconstruction from sparse data

  • Pontón Martínez, José Luis|||0000-0001-6576-4528
  • Yun, Haoran
  • Aristidou, Andreas
  • Andújar Gran, Carlos Antonio|||0000-0002-8480-4713
  • Pelechano Gómez, Núria|||0000-0002-1437-245X
Accurate and reliable human motion reconstruction is crucial for creating natural interactions of full-body avatars in Virtual Reality (VR) and entertainment applications. As the Metaverse and social applications gain popularity, users are seeking cost-effective solutions to create full-body animations that are comparable in quality to those produced by commercial motion capture systems. In order to provide affordable solutions though, it is important to minimize the number of sensors attached to the subject’s body. Unfortunately, reconstructing the full-body pose from sparse data is a heavily under-determined problem. Some studies that use IMU sensors face challenges in reconstructing the pose due to positional drift and ambiguity of the poses. In recent years, some mainstream VR systems have released 6-degree-of-freedom (6-DoF) tracking devices providing positional and rotational information. Nevertheless, most solutions for reconstructing full-body poses rely on traditional inverse kinematics (IK) solutions, which often produce non-continuous and unnatural poses. In this article, we introduce SparsePoser, a novel deep learning-based solution for reconstructing a full-body pose from a reduced set of six tracking devices. Our system incorporates a convolutional-based autoencoder that synthesizes high-quality continuous human poses by learning the human motion manifold from motion capture data. Then, we employ a learned IK component, made of multiple lightweight feed-forward neural networks, to adjust the hands and feet toward the corresponding trackers. We extensively evaluate our method on publicly available motion capture datasets and with real-time live demos. We show that our method outperforms state-of-the-art techniques using IMU sensors or 6-DoF tracking devices, and can be used for users with different body dimensions and proportions., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie SkŁodowska-Curie grant agreement No 860768 (CLIPE project), HORIZON-CL4-2022-HUMAN-01 grant agreement No 101093159 (XR4ED), and from MCIN/AEI/10.13039/501100011033/FEDER, UE (PID2021-122136OB-C21). This project has also received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy. Jose Luis Ponton was also funded by the Spanish Ministry of Universities (FPU21/01927)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Simulating real-life scenarios to better understand the spread of diseases under different contexts

  • Blanco Guerra, Rafael Andrés
  • Patow, Gustavo Ariel|||0000-0002-1977-9101
  • Pelechano Gómez, Núria|||0000-0002-1437-245X
Current statistical models to simulate pandemics miss the most relevant information about the close atomic interactions between individuals which is the key aspect of virus spread. Thus, they lack a proper visualization of such interactions and their impact on virus spread. In the field of computer graphics, and more specifically in computer animation, there have been many crowd simulation models to populate virtual environments. However, the focus has typically been to simulate reasonable paths between random or semi-random locations in a map, without any possibility of analyzing specific individual behavior. We propose a crowd simulation framework to accurately simulate the interactions in a city environment at the individual level, with the purpose of recording and analyzing the spread of human diseases. By simulating the whereabouts of agents throughout the day by mimicking the actual activities of a population in their daily routines, we can accurately predict the location and duration of interactions between individuals, thus having a model that can reproduce the spread of the virus due to human-to-human contact. Our results show the potential of our framework to closely simulate the virus spread based on real agent-to-agent contacts. We believe that this could become a powerful tool for policymakers to make informed decisions in future pandemics and to better communicate the impact of such decisions to the general public., This work has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska Curie grant agreement No 860768 (CLIPE project). Also, it was partially funded by grants PID2021-122136OB- C21 and PID2021-122136OB-C22 funded by MCIN/AEI/https://doi.org/10.13039/ 50110 0011033 and by ERDF A way of making Europe., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Animation fidelity in self-avatars: impact on user performance and sense of agency

  • Yun, Haoran
  • Pontón Martínez, José Luis|||0000-0001-6576-4528
  • Andújar Gran, Carlos Antonio|||0000-0002-8480-4713
  • Pelechano Gómez, Núria|||0000-0002-1437-245X
The use of self-avatars is gaining popularity thanks to affordable VR headsets. Unfortunately, mainstream VR devices often use a small number of trackers and provide low-accuracy animations. Previous studies have shown that the Sense of Embodiment, and in particular the Sense of Agency, depends on the extent to which the avatar's movements mimic the user's movements. However, few works study such effect for tasks requiring a precise interaction with the environment, i.e., tasks that require accurate manipulation, precise foot stepping, or correct body poses. In these cases, users are likely to notice inconsistencies between their self-avatars and their actual pose. In this paper, we study the impact of the animation fidelity of the user avatar on a variety of tasks that focus on arm movement, leg movement and body posture. We compare three different animation techniques: two of them using Inverse Kinematics to reconstruct the pose from sparse input (6 trackers), and a third one using a professional motion capture system with 17 inertial sensors. We evaluate these animation techniques both quantitatively (completion time, unintentional collisions, pose accuracy) and qualitatively (Sense of Embodiment). Our results show that the animation quality affects the Sense of Embodiment. Inertial-based MoCap performs significantly better in mimicking body poses. Surprisingly, IK-based solutions using fewer sensors outperformed MoCap in tasks requiring accurate positioning, which we attribute to the higher latency and the positional drift that causes errors at the end-effectors, which are more noticeable in contact areas such as the feet., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 860768 (CLIPE project) and from MCIN/AEI/10.13039/501100011033/FEDER, UE (PID2021-122136OB-C21). Jose Luis Ponton was also funded by the Spanish Ministry of Universities (FPU21/01927)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Combining motion matching and orientation prediction to animate avatars for consumer-grade VR devices

  • Pontón Martínez, José Luis|||0000-0001-6576-4528
  • Yun, Haoran
  • Andújar Gran, Carlos Antonio|||0000-0002-8480-4713
  • Pelechano Gómez, Núria|||0000-0002-1437-245X
The animation of user avatars plays a crucial role in conveying their pose, gestures, and relative distances to virtual objects or other users. Self-avatar animation in immersive VR helps improve the user experience and provides a Sense of Embodiment. However, consumer-grade VR devices typically include at most three trackers, one at the Head Mounted Display (HMD), and two at the handheld VR controllers. Since the problem of reconstructing the user pose from such sparse data is ill-defined, especially for the lower body, the approach adopted by most VR games consists of assuming the body orientation matches that of the HMD, and applying animation blending and time-warping from a reduced set of animations. Unfortunately, this approach produces noticeable mismatches between user and avatar movements. In this work we present a new approach to animate user avatars that is suitable for current mainstream VR devices. First, we use a neural network to estimate the user's body orientation based on the tracking information from the HMD and the hand controllers. Then we use this orientation together with the velocity and rotation of the HMD to build a feature vector that feeds a Motion Matching algorithm. We built a MoCap database with animations of VR users wearing a HMD and used it to test our approach on both self-avatars and other users’ avatars. Our results show that our system can provide a large variety of lower body animations while correctly matching the user orientation, which in turn allows us to represent not only forward movements but also stepping in any direction., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 860768 (CLIPE project) and the Spanish Ministry of Science and Innovation (PID2021-122136OB-C21)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

MPACT: Mesoscopic profiling and abstraction of crowd trajectories

  • Lemonari, Marilena
  • Panayiotou, Andreas|||0000-0002-7614-1059
  • Kyriakou, Theodoros|||0000-0003-0699-2445
  • Pelechano Gómez, Núria|||0000-0002-1437-245X
  • Chrysanthou, Yiorgos
  • Aristidou, Andreas
  • Charalambous, Panayiotis
Simulating believable crowds for applications like movies or games is challenging due to the many components that comprise a realistic outcome. Users typically need to manually tune a large number of simulation parameters until they reach the desired results. We introduce MPACT, a framework that leverages image-based encoding to convert unlabelled crowd data into meaningful and controllable parameters for crowd generation. In essence, we train a parameter prediction network on a diverse set of synthetic data, which includes pairs of images and corresponding crowd profiles. The learned parameter space enables: (a) implicit crowd authoring and control, allowing users to define desired crowd scenarios using real-world trajectory data, and (b) crowd analysis, facilitating the identification of crowd behaviours in the input and the classification of unseen scenarios through operations within the latent space. We quantitatively and qualitatively evaluate our framework, comparing it against real-world data and selected baselines, while also conducting user studies with expert and novice users. Our experiments show that the generated crowds score high in terms of simulation believability, plausibility and crowd behaviour faithfulness., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No 860768 (CLIPE project).This work has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578, the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy and Department of Research and Universities of the Government of Catalonia. Grant Number: 2021 SGR 01035. This work has received funding from the European Union under grant agreement No 10192889., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

TRAIL: Simulating the impact of human locomotion on natural landscapes

  • Alvarado Pinero, Eduardo
  • Argudo Medrano, Óscar|||0000-0003-3943-1839
  • Rohmer, Damien
  • Cani, Marie-Paule
  • Pelechano Gómez, Núria|||0000-0002-1437-245X
Human and animal presence in natural landscapes is initially revealed by the immediate impact of their locomotion, from footprints to crushed grass. In this work, we present an approach to model the effects of virtual characters on natural terrains, focusing on the impact of human locomotion. We introduce a lightweight solution to compute accurate foot placement on uneven ground and infer dynamic foot pressure from kinematic animation data and the mass of the character. A ground and vegetation model enables us to effectively simulate the local impact of locomotion on soft soils and plants over time, resulting in the formation of visible paths. As our results show, we can parameterize various soil materials and vegetation types validated with real-world data. Our method can be used to significantly increase the realism of populated natural landscapes and the sense of presence in virtual applications and games., This work is part of the project SENDA (TED2021-129761B-I00), funded by MCIN/AEI/10.13039/501100011033 and the European Union “NextGenerationEU”/PRTR. It is also part of the European Union’s Horizon 2020 research and innovation programme CLIPE, under the Marie Skłodowska-Curie grant agreement No. 860768., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Exploring the role of expected collision feedback in crowded virtual environments

  • Yun, Haoran
  • Pontón Martínez, José Luis
  • Beacco Porres, Alejandro
  • Andújar Gran, Carlos Antonio
  • Pelechano Gómez, Núria
An increasing number of virtual reality applications require environments that emulate real-world conditions. These environments often involve dynamic virtual humans showing realistic behaviors. Understanding user perception and navigation among these virtual agents is key for designing realistic and effective environments featuring groups of virtual humans. While collision risk significantly influences human locomotion in the real world, this risk is largely absent in virtual settings. This paper studies the impact of the expected collision feedback on user perception and interaction with virtual crowds. We examine the effectiveness of commonly used collision feedback techniques (auditory cues and tactile vibrations) as well as inducing participants to expect that a physical bump with a real person might occur, as if some virtual humans actually correspond to real persons embodied into them and sharing the same physical space. Our results indicate that the expected collision feedback significantly influences both participant behavior—encompassing global navigation and local movements—and subjective perceptions of presence and copresence. Specifically, the introduction of a perceived risk of actual collision was found to significantly impact global navigation strategies and increase the sense of presence. Auditory cues had a similar effect on global navigation and additionally enhanced the sense of copresence. In contrast, vibrotactile feedback was primarily effective in influencing local movements., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860768 (CLIPE project), and from MCIN/AEI/10.13039/501100011033/FEDER, UE (PID2021- 122136OB-C21), ”A way to make Europe”. Jose Luis Ponton was also funded by the Spanish Ministry of Universities (FPU21/01927)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

TRAIL: Simulating the impact of human locomotion on natural landscapes

  • Alvarado Pinero, Eduardo
  • Argudo Medrano, Óscar
  • Rohmer, Damien
  • Cani, Marie-Paule
  • Pelechano Gómez, Núria
Human and animal presence in natural landscapes is initially revealed by the immediate impact of their locomotion, from footprints to crushed grass. In this work, we present an approach to model the effects of virtual characters on natural terrains, focusing on the impact of human locomotion. We introduce a lightweight solution to compute accurate foot placement on uneven ground and infer dynamic foot pressure from kinematic animation data and the mass of the character. A ground and vegetation model enables us to effectively simulate the local impact of locomotion on soft soils and plants over time, resulting in the formation of visible paths. As our results show, we can parameterize various soil materials and vegetation types validated with real-world data. Our method can be used to significantly increase the realism of populated natural landscapes and the sense of presence in virtual applications and games., This work is part of the project SENDA (TED2021-129761B-I00), funded by MCIN/AEI/10.13039/501100011033 and the European Union “NextGenerationEU”/PRTR. It is also part of the European Union’s Horizon 2020 research and innovation programme CLIPE, under the Marie Skłodowska-Curie grant agreement No. 860768., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

MPACT: Mesoscopic profiling and abstraction of crowd trajectories

  • Lemonari, Marilena
  • Panayiotou, Andreas
  • Kyriakou, Theodoros
  • Pelechano Gómez, Núria
  • Chrysanthou, Yiorgos
  • Aristidou, Andreas
  • Charalambous, Panayiotis
Simulating believable crowds for applications like movies or games is challenging due to the many components that comprise a realistic outcome. Users typically need to manually tune a large number of simulation parameters until they reach the desired results. We introduce MPACT, a framework that leverages image-based encoding to convert unlabelled crowd data into meaningful and controllable parameters for crowd generation. In essence, we train a parameter prediction network on a diverse set of synthetic data, which includes pairs of images and corresponding crowd profiles. The learned parameter space enables: (a) implicit crowd authoring and control, allowing users to define desired crowd scenarios using real-world trajectory data, and (b) crowd analysis, facilitating the identification of crowd behaviours in the input and the classification of unseen scenarios through operations within the latent space. We quantitatively and qualitatively evaluate our framework, comparing it against real-world data and selected baselines, while also conducting user studies with expert and novice users. Our experiments show that the generated crowds score high in terms of simulation believability, plausibility and crowd behaviour faithfulness., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No 860768 (CLIPE project).This work has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578, the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy and Department of Research and Universities of the Government of Catalonia. Grant Number: 2021 SGR 01035. This work has received funding from the European Union under grant agreement No 10192889., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Animation fidelity in self-avatars: impact on user performance and sense of agency

  • Yun, Haoran
  • Pontón Martínez, José Luis
  • Andújar Gran, Carlos Antonio
  • Pelechano Gómez, Núria
The use of self-avatars is gaining popularity thanks to affordable VR headsets. Unfortunately, mainstream VR devices often use a small number of trackers and provide low-accuracy animations. Previous studies have shown that the Sense of Embodiment, and in particular the Sense of Agency, depends on the extent to which the avatar's movements mimic the user's movements. However, few works study such effect for tasks requiring a precise interaction with the environment, i.e., tasks that require accurate manipulation, precise foot stepping, or correct body poses. In these cases, users are likely to notice inconsistencies between their self-avatars and their actual pose. In this paper, we study the impact of the animation fidelity of the user avatar on a variety of tasks that focus on arm movement, leg movement and body posture. We compare three different animation techniques: two of them using Inverse Kinematics to reconstruct the pose from sparse input (6 trackers), and a third one using a professional motion capture system with 17 inertial sensors. We evaluate these animation techniques both quantitatively (completion time, unintentional collisions, pose accuracy) and qualitatively (Sense of Embodiment). Our results show that the animation quality affects the Sense of Embodiment. Inertial-based MoCap performs significantly better in mimicking body poses. Surprisingly, IK-based solutions using fewer sensors outperformed MoCap in tasks requiring accurate positioning, which we attribute to the higher latency and the positional drift that causes errors at the end-effectors, which are more noticeable in contact areas such as the feet., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 860768 (CLIPE project) and from MCIN/AEI/10.13039/501100011033/FEDER, UE (PID2021-122136OB-C21). Jose Luis Ponton was also funded by the Spanish Ministry of Universities (FPU21/01927)., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

SparsePoser: Real-time full-body motion reconstruction from sparse data

  • Pontón Martínez, José Luis
  • Yun, Haoran
  • Aristidou, Andreas
  • Andújar Gran, Carlos Antonio
  • Pelechano Gómez, Núria
Accurate and reliable human motion reconstruction is crucial for creating natural interactions of full-body avatars in Virtual Reality (VR) and entertainment applications. As the Metaverse and social applications gain popularity, users are seeking cost-effective solutions to create full-body animations that are comparable in quality to those produced by commercial motion capture systems. In order to provide affordable solutions though, it is important to minimize the number of sensors attached to the subject’s body. Unfortunately, reconstructing the full-body pose from sparse data is a heavily under-determined problem. Some studies that use IMU sensors face challenges in reconstructing the pose due to positional drift and ambiguity of the poses. In recent years, some mainstream VR systems have released 6-degree-of-freedom (6-DoF) tracking devices providing positional and rotational information. Nevertheless, most solutions for reconstructing full-body poses rely on traditional inverse kinematics (IK) solutions, which often produce non-continuous and unnatural poses. In this article, we introduce SparsePoser, a novel deep learning-based solution for reconstructing a full-body pose from a reduced set of six tracking devices. Our system incorporates a convolutional-based autoencoder that synthesizes high-quality continuous human poses by learning the human motion manifold from motion capture data. Then, we employ a learned IK component, made of multiple lightweight feed-forward neural networks, to adjust the hands and feet toward the corresponding trackers. We extensively evaluate our method on publicly available motion capture datasets and with real-time live demos. We show that our method outperforms state-of-the-art techniques using IMU sensors or 6-DoF tracking devices, and can be used for users with different body dimensions and proportions., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie SkŁodowska-Curie grant agreement No 860768 (CLIPE project), HORIZON-CL4-2022-HUMAN-01 grant agreement No 101093159 (XR4ED), and from MCIN/AEI/10.13039/501100011033/FEDER, UE (PID2021-122136OB-C21). This project has also received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy. Jose Luis Ponton was also funded by the Spanish Ministry of Universities (FPU21/01927)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Authoring virtual crowds: a survey

  • Lemonari, Marilena
  • Blanco Guerra, Rafael Andrés
  • Charalambous, Panayiotis
  • Pelechano Gómez, Núria
  • Avraamides, Marios
  • Pettré, Julien
  • Chrysanthou, Yiorgos
Recent advancements in crowd simulation unravel a wide range of functionalities for virtual agents, delivering highly-realistic,natural virtual crowds. Such systems are of particular importance to a variety of applications in fields such as: entertainment(e.g., movies, computer games); architectural and urban planning; and simulations for sports and training. However, providingtheir capabilities to untrained users necessitates the development of authoring frameworks. Authoring virtual crowds is acomplex and multi-level task, varying from assuming control and assisting users to realise their creative intents, to deliveringintuitive and easy to use interfaces, facilitating such control. In this paper, we present a categorisation of the authorable crowdsimulation components, ranging from high-level behaviours and path-planning to local movements, as well as animation andvisualisation. We provide a review of the most relevant methods in each area, emphasising the amount and nature of influencethat the users have over the final result. Moreover, we discuss the currently available authoring tools (e.g., graphical userinterfaces, drag-and-drop), identifying the trends of early and recent work. Finally, we suggest promising directions for futureresearch that mainly stem from the rise of learning-based methods, and the need for a unified authoring framework., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No 860768 (CLIPE project). This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital Policy, Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Simulating real-life scenarios to better understand the spread of diseases under different contexts

  • Blanco Guerra, Rafael Andrés
  • Patow, Gustavo Ariel
  • Pelechano Gómez, Núria
Current statistical models to simulate pandemics miss the most relevant information about the close atomic interactions between individuals which is the key aspect of virus spread. Thus, they lack a proper visualization of such interactions and their impact on virus spread. In the field of computer graphics, and more specifically in computer animation, there have been many crowd simulation models to populate virtual environments. However, the focus has typically been to simulate reasonable paths between random or semi-random locations in a map, without any possibility of analyzing specific individual behavior. We propose a crowd simulation framework to accurately simulate the interactions in a city environment at the individual level, with the purpose of recording and analyzing the spread of human diseases. By simulating the whereabouts of agents throughout the day by mimicking the actual activities of a population in their daily routines, we can accurately predict the location and duration of interactions between individuals, thus having a model that can reproduce the spread of the virus due to human-to-human contact. Our results show the potential of our framework to closely simulate the virus spread based on real agent-to-agent contacts. We believe that this could become a powerful tool for policymakers to make informed decisions in future pandemics and to better communicate the impact of such decisions to the general public., This work has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska Curie grant agreement No 860768 (CLIPE project). Also, it was partially funded by grants PID2021-122136OB- C21 and PID2021-122136OB-C22 funded by MCIN/AEI/https://doi.org/10.13039/ 50110 0011033 and by ERDF A way of making Europe., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Social crowd simulation: improving realism with social rules and gaze behavior

  • Itatani, Reiya
  • Pelechano Gómez, Núria
Current crowd simulation models focus mostly on steering towards a goal while avoiding collisions based on the agent’s direction of movement. This leads to robot-like simulations since agents’ appear to always have their attention perfectly aligned with the direction of movement. In the real world, we observe that humans move in a crowd performing collision avoidance driven by attention, gaze, and non-verbal coordination with incoming traffic. In addition, humans exhibit different steering strategies based on whether they walk alone or in a group, whether they can look ahead and plan their best local movement, or react more abruptly because their gaze diverts from their direction of movement. Human gaze can be driven by movement, but also by distractions such as being engaged in conversation with other people or using mobile phones. These human features are overlooked in crowd simulation, often leading to perfectly smooth local movements of individuals. Unfortunately, this lack of social behaviour and variety in animations may be perceived as unrealistic when observing the results on a 2D display, and it may become even more apparent in immersive scenarios where the participant is at eye level with the virtual humans. This paper proposes a novel approach to enhance the realism of a rule-based crowd simulation model by incorporating social rules and gaze-driven attention with consistent animations. The ultimate goal is to make immersive virtual crowds more realistic. Our proposed method enhances existing crowd simulation frameworks by integrating social behavior models that affect both individual and collective dynamics, and drives gaze behavior to better simulate attention. We conducted validation user studies on both a 2D display and in immersive VR, and observed that applying these models to both the steering and animation levels significantly improves the realism of the crowd simulation. The 2D display based user study based on video comparisons showed that our model was perceived as more realistic and consistent with social behaviors compared to traditional collision avoidance approaches, that used only locomotion or random animations. The immersive user study showed that participants effectively detected the social behaviours included in our model as intended. The results revealed significant differences in the participants’ perceptions of the various behaviours exhibited by our social crowd model., This work was supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 860768 (CLIPE project), and by MCIN/AEI/10.13039/501100011033/FEDER, UE (PID2021-122136OBC21)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Combining motion matching and orientation prediction to animate avatars for consumer-grade VR devices

  • Pontón Martínez, José Luis
  • Yun, Haoran
  • Andújar Gran, Carlos Antonio
  • Pelechano Gómez, Núria
The animation of user avatars plays a crucial role in conveying their pose, gestures, and relative distances to virtual objects or other users. Self-avatar animation in immersive VR helps improve the user experience and provides a Sense of Embodiment. However, consumer-grade VR devices typically include at most three trackers, one at the Head Mounted Display (HMD), and two at the handheld VR controllers. Since the problem of reconstructing the user pose from such sparse data is ill-defined, especially for the lower body, the approach adopted by most VR games consists of assuming the body orientation matches that of the HMD, and applying animation blending and time-warping from a reduced set of animations. Unfortunately, this approach produces noticeable mismatches between user and avatar movements. In this work we present a new approach to animate user avatars that is suitable for current mainstream VR devices. First, we use a neural network to estimate the user's body orientation based on the tracking information from the HMD and the hand controllers. Then we use this orientation together with the velocity and rotation of the HMD to build a feature vector that feeds a Motion Matching algorithm. We built a MoCap database with animations of VR users wearing a HMD and used it to test our approach on both self-avatars and other users’ avatars. Our results show that our system can provide a large variety of lower body animations while correctly matching the user orientation, which in turn allows us to represent not only forward movements but also stepping in any direction., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 860768 (CLIPE project) and the Spanish Ministry of Science and Innovation (PID2021-122136OB-C21)., Peer Reviewed, Postprint (published version)