Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 4
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

A multimodal late fusion framework for physiological sensor and audio-signal-based stress detection: an experimental study and public dataset

  • Xefteris, Vasileios-Rafail
  • Domínguez Bajo, Mónica
  • Grivolla, Jens
  • Tsanousa, Athina
  • Zaffanela, Francesco
  • Monego, Martina
  • Symeonidis, Spyridon
  • Diplaris, Sotiris
  • Wanner, Leo
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Stress can be considered a mental/physiological reaction in conditions of high discomfort and challenging situations. The levels of stress can be reflected in both the physiological responses and speech signals of a person. Therefore the study of the fusion of the two modalities is of great interest. For this cause, public datasets are necessary so that the different proposed solutions can be comparable. In this work, a publicly available multimodal dataset for stress detection is introduced, including physiological signals and speech cues data. The physiological signals include electrocardiograph (ECG), respiration (RSP), and inertial measurement unit (IMU) sensors equipped in a smart vest. A data collection protocol was introduced to receive physiological and audio data based on alterations between well-known stressors and relaxation moments. Five subjects participated in the data collection, where both their physiological and audio signals were recorded by utilizing the developed smart vest and audio recording application. In addition, an analysis of the data and a decision-level fusion scheme is proposed. The analysis of physiological signals includes a massive feature extraction along with various fusion and feature selection methods. The audio analysis comprises a state-of-the-art feature extraction fed to a classifier to predict stress levels. Results from the analysis of audio and physiological signals are fused at a decision level for the final stress level detection, utilizing a machine learning algorithm. The whole framework was also tested in a real-life pilot scenario of disaster management, where users were acting as first responders while their stress was monitored in real time., This work was supported by the XR4DRAMA project funded by the European Commission (H2020) under the grant number 952133.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

How much pretraining data do language models need to learn syntax?

  • Pérez-Mayos, Laura
  • Ballesteros, Miguel
  • Wanner, Leo
Comunicació presentada a 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021), celebrat del 7 a l'11 de novembre de 2021 de manera virtual., Transformers-based pretrained language models achieve outstanding results in many wellknown NLU benchmarks. However, while pretraining methods are very convenient, they are expensive in terms of time and resources. This calls for a study of the impact of pretraining data size on the knowledge of the models. We explore this impact on the syntactic capabilities of RoBERTa, using models trained on incremental sizes of raw text data. First, we use syntactic structural probes to determine whether models pretrained on more data encode a higher amount of syntactic information. Second, we perform a targeted syntactic evaluation to analyze the impact of pretraining data size on the syntactic generalization performance of the models. Third, we compare the performance of the different models on three downstream applications: part-of-speech tagging, dependency parsing and paraphrase identification. We complement our study with an analysis of the cost-benefit trade-off of training such models. Our experiments show that while models pretrained on more data encode more syntactic knowledge and perform better on downstream applications, they do not always offer a better performance across the different syntactic phenomena and come at a higher financial and environmental cost., This work has been partially funded by the European Commission via its H2020 Research Program under the contract numbers 786731, 825079, 870930, and 952133. This work has been partially supported by the ICT PhD program of Universitat Pompeu Fabra through a travel grant.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A multimodal late fusion framework for physiological sensor and audio-signal-based stress detection: an experimental study and public dataset

  • Xefteris, Vasileios-Rafail
  • Domínguez Bajo, Mónica
  • Grivolla, Jens
  • Tsanousa, Athina
  • Zaffanela, Francesco
  • Monego, Martina
  • Symeonidis, Spyridon
  • Diplaris, Sotiris
  • Wanner, Leo
  • Vrochidis, Stefanos
  • Kompatsiaris, Ioannis
Stress can be considered a mental/physiological reaction in conditions of high discomfort and challenging situations. The levels of stress can be reflected in both the physiological responses and speech signals of a person. Therefore the study of the fusion of the two modalities is of great interest. For this cause, public datasets are necessary so that the different proposed solutions can be comparable. In this work, a publicly available multimodal dataset for stress detection is introduced, including physiological signals and speech cues data. The physiological signals include electrocardiograph (ECG), respiration (RSP), and inertial measurement unit (IMU) sensors equipped in a smart vest. A data collection protocol was introduced to receive physiological and audio data based on alterations between well-known stressors and relaxation moments. Five subjects participated in the data collection, where both their physiological and audio signals were recorded by utilizing the developed smart vest and audio recording application. In addition, an analysis of the data and a decision-level fusion scheme is proposed. The analysis of physiological signals includes a massive feature extraction along with various fusion and feature selection methods. The audio analysis comprises a state-of-the-art feature extraction fed to a classifier to predict stress levels. Results from the analysis of audio and physiological signals are fused at a decision level for the final stress level detection, utilizing a machine learning algorithm. The whole framework was also tested in a real-life pilot scenario of disaster management, where users were acting as first responders while their stress was monitored in real time., This work was supported by the XR4DRAMA project funded by the European Commission (H2020) under the grant number 952133.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

How much pretraining data do language models need to learn syntax?

  • Pérez-Mayos, Laura
  • Ballesteros, Miguel
  • Wanner, Leo
Comunicació presentada a 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021), celebrat del 7 a l'11 de novembre de 2021 de manera virtual., Transformers-based pretrained language models achieve outstanding results in many wellknown NLU benchmarks. However, while pretraining methods are very convenient, they are expensive in terms of time and resources. This calls for a study of the impact of pretraining data size on the knowledge of the models. We explore this impact on the syntactic capabilities of RoBERTa, using models trained on incremental sizes of raw text data. First, we use syntactic structural probes to determine whether models pretrained on more data encode a higher amount of syntactic information. Second, we perform a targeted syntactic evaluation to analyze the impact of pretraining data size on the syntactic generalization performance of the models. Third, we compare the performance of the different models on three downstream applications: part-of-speech tagging, dependency parsing and paraphrase identification. We complement our study with an analysis of the cost-benefit trade-off of training such models. Our experiments show that while models pretrained on more data encode more syntactic knowledge and perform better on downstream applications, they do not always offer a better performance across the different syntactic phenomena and come at a higher financial and environmental cost., This work has been partially funded by the European Commission via its H2020 Research Program under the contract numbers 786731, 825079, 870930, and 952133. This work has been partially supported by the ICT PhD program of Universitat Pompeu Fabra through a travel grant.