Skip to main content

INTELIGENCIA ARTIFICIAL DE SOPORTE A EXPERIENCIAS MUSICALES: HACIA UNA APROXIMACION BASADA EN DATOS Y CENTRADA EN EL USUARIO

PID2019-111403GB-I00

•
Funding agency name Agencia Estatal de Investigación
Funding agency acronym AEI
Program Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Call Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Call year 2019
State Plan 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Beneficiary centre UNIVERSITAT POMPEU FABRA CCT

Resultados relacionados

Resultados totales (Incluyendo duplicados): 20
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Audio and music analysis on the web using Essentia.js

  • Correya, Albin Andrew
  • Marcos Fernández, Jorge
  • Joglar-Ongay, Luis
  • Alonso Jiménez, Pablo
  • Serra, Xavier
  • Bogdanov, Dmitry
Open-source software libraries have a significant impact on the development of Audio Signal Processing and Music Information Retrieval (MIR) systems. Despite the abundance of such tools, there is a lack of an extensive and easy-to-use reference library for audio feature extraction on Web clients. In this article, we present Essentia.js, an open-source JavaScript (JS) library for audio and music analysis on both web clients and JS engines. Along with the Web Audio API, it can be used for both offline and real-time audio feature extraction on web browsers. Essentia.js is modular, lightweight, and easy-to-use, deploy, maintain, and integrate into the existing plethora of JS libraries and web technologies. It is powered by a WebAssembly back end cross-compiled from the Essentia C++ library, which facilitates a JS interface to a wide range of low-level and high-level audio features, including signal processing MIR algorithms as well as pre-trained TensorFlow.js machine learning models. It also provides a higher-level JS API and add-on MIR utility modules along with extensive documentation, usage examples, and tutorials. We benchmark the proposed library on two popular web browsers and the Node.js engine, and four devices, including mobile Android and iOS, comparing it to the native performance of Essentia and the Meyda JS library., The work on Essentia.js has been partially funded by the Ministry of Science and Innovation of the Spanish Government under the grant agreement PID2019-111403GB-I00 (Musical AI).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Voice assignment in vocal quartets using deep learning models based on pitch salience

  • Cuesta, Helena
  • Gómez Gutiérrez, Emilia, 1975-
This paper deals with the automatic transcription of four-part, a cappella singing, audio performances. In particular, we exploit an existing, deep-learning based, multiple F0 estimation method and complement it with two neural network architectures for voice assignment (VA) in order to create a music transcription system that converts an input audio mixture into four pitch contours. To train our VA models, we create a novel synthetic dataset by collecting 5381 choral music scores from public-domain music archives, which we make publicly available for further research. We compare the performance of the proposed VA models on different types of input data, as well as to a hidden Markov model-based baseline system. In addition, we assess the generalization capabilities of these models on audio recordings with differing pitch distributions and vocal music styles. Our experiments show that the two proposed models, a CNN and a ConvLSTM, have very similar performance, and both of them outperform the baseline HMM-based system. We also observe a high confusion rate between the alto and tenor voice parts, which commonly have overlapping pitch ranges, while the bass voice has the highest scores in all evaluated scenarios., This work is partially supported by the European Commission under the TROMPA project (H2020 770376), the Spanish Ministry of Science and Innovation under the Musical AI project (PID2019-111403GB-I00), and by AGAUR (Generalitat de Catalunya) through an FI Predoctoral Grant (2018FI-B01015).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Repertoire-specific vocal pitch data generation for improved melodic analysis of carnatic music

  • Plaja-Roglans, Genís
  • Nutall, Thomas
  • Pearson, Lara
  • Serra, Xavier
  • Miron, Marius
Deep Learning methods achieve state-of-the-art in many tasks, including vocal pitch extraction. However, these methods rely on the availability of pitch track annotations without errors, which are scarce and expensive to obtain for Carnatic Music. Here we identify the tradition-related challenges and propose tailored solutions to generate a novel, large, and open dataset, the Saraga-Carnatic-Melody-Synth (SCMS), comprising audio mixtures and time-aligned vocal pitch annotations. Through a cross-cultural evaluation leveraging this novel dataset, we show improvements in the performance of Deep Learning vocal pitch extraction methods on Indian Art Music recordings. Additional experiments show that the trained models outperform the currently used heuristic-based pitch extraction solutions for the computational melodic analysis of Carnatic Music and that this improvement leads to better results in the musicologically relevant task of repeated melodic pattern discovery when evaluated using expert annotations. The code and annotations are made available for reproducibility. The novel dataset and trained models are also integrated into the Python package compIAM1 which allows them to be used out-of-the-box., This work is funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI) within the Musical AI Project – PID2019-111403GB-I00/AEI/10.13039/501100011033.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Personalisation and profiling using algorithms and not-so-popular Colombian music: goal-directed mechanisms in music emotion recognition

  • Gómez-Cañon, Juan Sebastián
  • Lennie, Thomas Magnus
  • Eerola, Tuomas
  • Aragón, Pablo
  • Cano, Estefanía
  • Herrera, Perfecto
  • Gómez, Emilia
This work investigates how personalised Music Emotion Recognition (MER) systems may lead to sensitive profiling when applied to musically induced emotions in politically charged contexts. We focus on traditional Colombian music with explicit political content, including (1) vallenatos and social songs aligned with the left-wing guerrilla Fuerzas Armadas Revolucionarias de Colombia (FARC), and (2) corridos linked to sympathisers of the right-wing paramilitary group Autodefensas Unidas de Colombia (AUC). Using data from 49 participants with diverse political leanings, we train personalised machine learning models to predict induced emotional responses -particularly negative emotions. Our findings reveal that political identity plays a significant role in shaping emotional experiences of music with explicit political content, and that emotion recognition models can capture this variation to a certain extent. These results raise critical concerns about the potential misuse of emotion recognition technologies. What is often framed as a tool for wellbeing and emotional regulation could, in politically sensitive contexts, be repurposed for user profiling. This work highlights the ethical risks of deploying AI-driven emotion analysis without safeguards, particularly among populations that are politically or socially vulnerable. We argue that subjective emotional responses may constitute sensitive personal data, and that failing to account for their sociopolitical context could amplify harm and exclusion., JSGC's contribution was partially supported by the European Commission through the TROMPA project (H2020 770376) and the project Musical AI (PID2019-111403GB-I00/AEI/10.13039/501100011033), funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI). TML was supported by the Arts and Humanities Research Council (AHRC) through a Northern Bridge Doctoral Training Partnership scholarship (grant code: AH/L503927/1).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

A deep-learning based framework for source separation, analysis, and synthesis of choral ensembles

  • Chandna, Pritish
  • Cuesta, Helena
  • Petermann, Darius
  • Gómez Gutiérrez, Emilia, 1975-
Choral singing in the soprano, alto, tenor and bass (SATB) format is a widely practiced and studied art form with significant cultural importance. Despite the popularity of the choral setting, it has received little attention in the field of Music Information Retrieval. However, the recent publication of high-quality choral singing datasets as well as recent developments in deep learning based methodologies applied to the field of music and speech processing, have opened new avenues for research in this field. In this paper, we use some of the publicly available choral singing datasets to train and evaluate state-of-the-art source separation algorithms from the speech and music domains for the case of choral singing. Furthermore, we evaluate existing monophonic F0 estimators on the separated unison stems and propose an approximation of the perceived F0 of a unison signal. Additionally, we present a set of applications combining the proposed methodologies, including synthesizing a single singer voice from the unison, and transposing and remixing the separated stems into a synthetic multi-singer choral signal. We finally conduct a set of listening tests to perform a perceptual evaluation of the results we obtain with the proposed methodologies., This work is partially supported by the European Commission under the TROMPA project (H2020 770376), and the project Musical AI (PID 2019-111403GB-I00/AEI/10.13039/501100011033) funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Applications of biological and physiological signals in commercial video gaming and game research: a review

  • Hughes, Alayna
  • Jordà Puig, Sergi
Video gaming is now available as a fully immersive experience that creates responsive inputs and outputs concerning the user, and some experimental developers have integrated the use of the voice, brain, or muscles as input controls. The use of physiological signal equipment can provide valuable information regarding the emotion of a player or patient during gameplay. In this article, we discuss five of the most common biosignals that are used in gaming research, and their function and devices that may be used for measurement. We break down those individual signals and present examples of research studies that implement them. We also discuss the usage of biological signals within commercial gaming and conclude with some possible future directions for the use of biological signals in gaming and game research., This research has been partially supported by the project Musical AI - PID2019-111403GB-I00/AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

The creative drummer: an eeg-based pilot study on the correlates of emotions and creative drum playing

  • Ramírez, Rafael, 1966-
  • Reija, Xavier
It is reasonable to assume that emotional processes are involved in creative tasks and the generation of creative ideas. In this pilot study, we investigate the emotional correlates in professional drummers during different degrees of creative music playing. Ten participants performed three tasks: repetitive rhythmic drum playing, pattern-based improvisation, and attention-intensive free improvisation, while their EEG activity was recorded. Arousal and valence levels were estimated from the EEG data at baseline and for the three tasks. Results show significantly increased levels of valence (i.e., increased prefrontal right alpha power compared to prefrontal left alpha power) during pattern-based and free improvisation relative to baseline, and significantly increased levels of valence during free improvisation relative to pattern-based improvisation. These results seem to indicate that positive emotion (characterized as increased valence) is associated with the creation of original ideas in drum playing and that the freer the creative process, the greater the positive effect. The implication of these results may be of particular relevance in the fields of music-based therapeutic interventions and music pedagogy., This work was partly sponsored by the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project) and by the Musical AI project—(PID2019-111403GB-I00/AEI/10.13039/501100011033).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Error monitoring and correction in violin performance: an EEG study

  • Blanco, Angel David
  • Costa-Faidella, Jordi
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Dalmazzo, David
  • Ramirez Melendez, Rafael
  • San Miguel Insúa, Iria
Previous research using electroencephalography (EEG) has investigated neural error-related music production processes in expert pianists reporting a frontal event-related negativity followed by a P300 event-related potential (ERP) after mistuned notes. However, piano playing does not rely on auditory feedback and corrective movements when dealing with accurate note tuning, thus offering an incomplete picture about error monitoring processes in players of many other musical instruments, including the human voice. We here designed a setup allowing us to collect the EEG of expert bow-string players while performing short melodies on the violin, (a fretless instrument) while also manipulating in real-time the produced pitch in a controlled manner. 15 expert bow-string players were asked to 1) listen to a reference melody of four notes; 2) play that melody with a violin (Active Condition); and 3) listen passively to the replay of their performance (Replayed Condition). Randomly, the auditory feedback of one of the four melody notes was manipulated in the Active Condition by lowering or lifting the pitch by half semitone. We found that mistuned notes, independently of whether they were manipulated externally or were produced by natural playing errors, elicited an f-ERN followed by a P300. We also found that the amplitude of the f-ERN was larger for faster corrective movements than for slower ones. In addition, between the f-ERN and the P300, we also identified two negative components-one in parietal areas and another in central-right regions-whose amplitudes were greater for slower corrective movements. We suggest the existence of two error-monitoring systems: a "fast" system mediated by the medial frontal cortex that quickly corrects errors when certainty is high and a "slow" system mediated by the posterior parietal cortex that accumulates evidence under uncertainty before triggering correction., This work was partly sponsored by the Musical AI project (PID2019-111403GB-I00) funded by MCIN/AEI/ 10.13039/501100011033 and by 'ERDF A way of making Europe'; by the Project PID2021-128790NB-I00, funded by MCIN/AEI/ 10.13039/501100011033 / FEDER UE; and by the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project). It has been additionally supported by the Excellence Research Group 2017SGR-974 funded by the Secretaria d'Universitats i Recerca del Departament d'Empresa i Coneixement de la Generalitat de Catalunya and by the MDM-2017-0729-18-2M Maria de Maeztu Center of Excellence UBNeuro, funded by MCIN/AEI/ 10.13039/501100011033. ISM was supported by grant RYC-2013-12577, funded by MCIN/AEI/ 10.13039/501100011033 and by 'ESF Investing in your future'. Open access publication fees were partly covered by the University of Barcelona (Ajudes de la Universitat de Barcelona per a publicar en accés obert).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

TROMPA-MER: an open dataset for personalized music emotion recognition

  • Gómez Cañón, Juan Sebastián
  • Gutiérrez Páez, Nicolás Felipe
  • Porcaro, Lorenzo
  • Porter, Alastair
  • Cano, Estefanía
  • Herrera Boyer, Perfecto, 1964-
  • Gkiokas, Aggelos
  • Santos Rodríguez, Patrícia
  • Hernández Leo, Davinia
  • Karreman, Casper
  • Gómez Gutiérrez, Emilia, 1975-
We present a platform and a dataset to help research on Music Emotion Recognition (MER). We developed the Music Enthusiasts platform aiming to improve the gathering and analysis of the so-called “ground truth” needed as input to MER systems. Firstly, our platform involves engaging participants using citizen science strategies and generate music emotion annotations – the platform presents didactic information and musical recommendations as incentivization, and collects data regarding demographics, mood, and language from each participant. Participants annotated each music excerpt with single free-text emotion words (in native language), distinct forced-choice emotion categories, preference, and familiarity. Additionally, participants stated the reasons for each annotation – including those distinctive of emotion perception and emotion induction. Secondly, our dataset was created for personalized MER and contains information from 181 participants, 4721 annotations, and 1161 music excerpts. To showcase the use of the dataset, we present a methodology for personalization of MER models based on active learning. The experiments show evidence that using the judgment of the crowd as prior knowledge for active learning allows for more effective personalization of MER systems for this particular dataset. Our dataset is publicly available and we invite researchers to use it for testing MER systems., This work was funded by the European Commission under the TROMPA project (H2020 770376) and the Project Musical AI - PID2019-111403GB-I00/AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Music-enhanced emotion identification of facial emotions in autistic spectrum disorder children: a pilot EEG study

  • Ramírez, Rafael, 1966-
  • Matamoros, Elisabet
  • Hernández-Leo, Davinia
  • Mirabel, Júlia
  • Sánchez, Elisabet
  • Escude, Núria
The Autistic Spectrum Disorder (ASD) is characterized by a difficulty in expressing and interpreting others’ emotions. In particular, people with ASD have difficulties when interpreting emotions encoded in facial expressions. In the past, music interventions have been shown to improve autistic individuals’ emotional and social skills. The present study describes a pilot study to explore the usefulness of music as a tool for improving autistic children’s emotion recognition in facial expressions. Twenty-five children (mean age = 8.8 y, SD = 1.24) with high-functioning ASD and normal hearing participated in the study consisting of four weekly sessions of 15 min each. Twenty-five participants were randomly divided into an experimental group (N = 14) and a control group (N = 11). During each session, participants in the experimental group were exposed to images of facial expressions for four emotions (happy, sad, angry, and fear). Images were shown in three conditions, with the second condition consisting of music of congruent emotion with the shown images. Participants in the control group were shown only images in all three conditions. For six participants in each group, EEG data were acquired during the sessions, and instantaneous emotional responses (arousal and valence values) were extracted from the EEG data. Inter- and intra-session emotion identification improvement was measured in terms of verbal response accuracy, and EEG response differences were analyzed. A comparison of the verbal responses of the experimental group pre- and post-intervention showed a significant (p = 0.001) average improvement in emotion identification accuracy responses of 26% (SD = 3.4). Furthermore, emotional responses of the experimental group at the end of the study showed a higher correlation with the emotional stimuli being presented, compared with their emotional responses at the beginning of the study. No similar verbal responses improvement or EEG-stimuli correlation was found in the control group. These results seem to indicate that music can be used to improve both emotion identification in facial expressions and emotion induction through facial stimuli in children with high-functioning ASD., This work was partly sponsored by the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project) and by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI) project Musical AI—PID2019-111403GB-I00/AEI/10.13039/501100011033.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Audio and music analysis on the web using Essentia.js

  • Correya, Albin Andrew
  • Marcos Fernández, Jorge
  • Joglar-Ongay, Luis
  • Alonso Jiménez, Pablo
  • Serra, Xavier
  • Bogdanov, Dmitry
Open-source software libraries have a significant impact on the development of Audio Signal Processing and Music Information Retrieval (MIR) systems. Despite the abundance of such tools, there is a lack of an extensive and easy-to-use reference library for audio feature extraction on Web clients. In this article, we present Essentia.js, an open-source JavaScript (JS) library for audio and music analysis on both web clients and JS engines. Along with the Web Audio API, it can be used for both offline and real-time audio feature extraction on web browsers. Essentia.js is modular, lightweight, and easy-to-use, deploy, maintain, and integrate into the existing plethora of JS libraries and web technologies. It is powered by a WebAssembly back end cross-compiled from the Essentia C++ library, which facilitates a JS interface to a wide range of low-level and high-level audio features, including signal processing MIR algorithms as well as pre-trained TensorFlow.js machine learning models. It also provides a higher-level JS API and add-on MIR utility modules along with extensive documentation, usage examples, and tutorials. We benchmark the proposed library on two popular web browsers and the Node.js engine, and four devices, including mobile Android and iOS, comparing it to the native performance of Essentia and the Meyda JS library., The work on Essentia.js has been partially funded by the Ministry of Science and Innovation of the Spanish Government under the grant agreement PID2019-111403GB-I00 (Musical AI).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Repertoire-specific vocal pitch data generation for improved melodic analysis of carnatic music

  • Plaja-Roglans, Genís
  • Nutall, Thomas
  • Pearson, Lara
  • Serra, Xavier
  • Miron, Marius
Deep Learning methods achieve state-of-the-art in many tasks, including vocal pitch extraction. However, these methods rely on the availability of pitch track annotations without errors, which are scarce and expensive to obtain for Carnatic Music. Here we identify the tradition-related challenges and propose tailored solutions to generate a novel, large, and open dataset, the Saraga-Carnatic-Melody-Synth (SCMS), comprising audio mixtures and time-aligned vocal pitch annotations. Through a cross-cultural evaluation leveraging this novel dataset, we show improvements in the performance of Deep Learning vocal pitch extraction methods on Indian Art Music recordings. Additional experiments show that the trained models outperform the currently used heuristic-based pitch extraction solutions for the computational melodic analysis of Carnatic Music and that this improvement leads to better results in the musicologically relevant task of repeated melodic pattern discovery when evaluated using expert annotations. The code and annotations are made available for reproducibility. The novel dataset and trained models are also integrated into the Python package compIAM1 which allows them to be used out-of-the-box., This work is funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI) within the Musical AI Project – PID2019-111403GB-I00/AEI/10.13039/501100011033.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

TROMPA-MER: an open dataset for personalized music emotion recognition

  • Gómez Cañón, Juan Sebastián
  • Gutiérrez Páez, Nicolás Felipe
  • Porcaro, Lorenzo
  • Porter, Alastair
  • Cano, Estefanía
  • Herrera Boyer, Perfecto, 1964-
  • Gkiokas, Aggelos
  • Santos Rodríguez, Patrícia
  • Hernández Leo, Davinia
  • Karreman, Casper
  • Gómez Gutiérrez, Emilia, 1975-
We present a platform and a dataset to help research on Music Emotion Recognition (MER). We developed the Music Enthusiasts platform aiming to improve the gathering and analysis of the so-called “ground truth” needed as input to MER systems. Firstly, our platform involves engaging participants using citizen science strategies and generate music emotion annotations – the platform presents didactic information and musical recommendations as incentivization, and collects data regarding demographics, mood, and language from each participant. Participants annotated each music excerpt with single free-text emotion words (in native language), distinct forced-choice emotion categories, preference, and familiarity. Additionally, participants stated the reasons for each annotation – including those distinctive of emotion perception and emotion induction. Secondly, our dataset was created for personalized MER and contains information from 181 participants, 4721 annotations, and 1161 music excerpts. To showcase the use of the dataset, we present a methodology for personalization of MER models based on active learning. The experiments show evidence that using the judgment of the crowd as prior knowledge for active learning allows for more effective personalization of MER systems for this particular dataset. Our dataset is publicly available and we invite researchers to use it for testing MER systems., This work was funded by the European Commission under the TROMPA project (H2020 770376) and the Project Musical AI - PID2019-111403GB-I00/AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

The creative drummer: an eeg-based pilot study on the correlates of emotions and creative drum playing

  • Ramírez, Rafael, 1966-
  • Reija, Xavier
It is reasonable to assume that emotional processes are involved in creative tasks and the generation of creative ideas. In this pilot study, we investigate the emotional correlates in professional drummers during different degrees of creative music playing. Ten participants performed three tasks: repetitive rhythmic drum playing, pattern-based improvisation, and attention-intensive free improvisation, while their EEG activity was recorded. Arousal and valence levels were estimated from the EEG data at baseline and for the three tasks. Results show significantly increased levels of valence (i.e., increased prefrontal right alpha power compared to prefrontal left alpha power) during pattern-based and free improvisation relative to baseline, and significantly increased levels of valence during free improvisation relative to pattern-based improvisation. These results seem to indicate that positive emotion (characterized as increased valence) is associated with the creation of original ideas in drum playing and that the freer the creative process, the greater the positive effect. The implication of these results may be of particular relevance in the fields of music-based therapeutic interventions and music pedagogy., This work was partly sponsored by the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project) and by the Musical AI project—(PID2019-111403GB-I00/AEI/10.13039/501100011033).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Personalisation and profiling using algorithms and not-so-popular Colombian music: goal-directed mechanisms in music emotion recognition

  • Gómez-Cañon, Juan Sebastián
  • Lennie, Thomas Magnus
  • Eerola, Tuomas
  • Aragón, Pablo
  • Cano, Estefanía
  • Herrera, Perfecto
  • Gómez, Emilia
This work investigates how personalised Music Emotion Recognition (MER) systems may lead to sensitive profiling when applied to musically induced emotions in politically charged contexts. We focus on traditional Colombian music with explicit political content, including (1) vallenatos and social songs aligned with the left-wing guerrilla Fuerzas Armadas Revolucionarias de Colombia (FARC), and (2) corridos linked to sympathisers of the right-wing paramilitary group Autodefensas Unidas de Colombia (AUC). Using data from 49 participants with diverse political leanings, we train personalised machine learning models to predict induced emotional responses -particularly negative emotions. Our findings reveal that political identity plays a significant role in shaping emotional experiences of music with explicit political content, and that emotion recognition models can capture this variation to a certain extent. These results raise critical concerns about the potential misuse of emotion recognition technologies. What is often framed as a tool for wellbeing and emotional regulation could, in politically sensitive contexts, be repurposed for user profiling. This work highlights the ethical risks of deploying AI-driven emotion analysis without safeguards, particularly among populations that are politically or socially vulnerable. We argue that subjective emotional responses may constitute sensitive personal data, and that failing to account for their sociopolitical context could amplify harm and exclusion., JSGC's contribution was partially supported by the European Commission through the TROMPA project (H2020 770376) and the project Musical AI (PID2019-111403GB-I00/AEI/10.13039/501100011033), funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI). TML was supported by the Arts and Humanities Research Council (AHRC) through a Northern Bridge Doctoral Training Partnership scholarship (grant code: AH/L503927/1).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Music-enhanced emotion identification of facial emotions in autistic spectrum disorder children: a pilot EEG study

  • Ramírez, Rafael, 1966-
  • Matamoros, Elisabet
  • Hernández-Leo, Davinia
  • Mirabel, Júlia
  • Sánchez, Elisabet
  • Escude, Núria
The Autistic Spectrum Disorder (ASD) is characterized by a difficulty in expressing and interpreting others’ emotions. In particular, people with ASD have difficulties when interpreting emotions encoded in facial expressions. In the past, music interventions have been shown to improve autistic individuals’ emotional and social skills. The present study describes a pilot study to explore the usefulness of music as a tool for improving autistic children’s emotion recognition in facial expressions. Twenty-five children (mean age = 8.8 y, SD = 1.24) with high-functioning ASD and normal hearing participated in the study consisting of four weekly sessions of 15 min each. Twenty-five participants were randomly divided into an experimental group (N = 14) and a control group (N = 11). During each session, participants in the experimental group were exposed to images of facial expressions for four emotions (happy, sad, angry, and fear). Images were shown in three conditions, with the second condition consisting of music of congruent emotion with the shown images. Participants in the control group were shown only images in all three conditions. For six participants in each group, EEG data were acquired during the sessions, and instantaneous emotional responses (arousal and valence values) were extracted from the EEG data. Inter- and intra-session emotion identification improvement was measured in terms of verbal response accuracy, and EEG response differences were analyzed. A comparison of the verbal responses of the experimental group pre- and post-intervention showed a significant (p = 0.001) average improvement in emotion identification accuracy responses of 26% (SD = 3.4). Furthermore, emotional responses of the experimental group at the end of the study showed a higher correlation with the emotional stimuli being presented, compared with their emotional responses at the beginning of the study. No similar verbal responses improvement or EEG-stimuli correlation was found in the control group. These results seem to indicate that music can be used to improve both emotion identification in facial expressions and emotion induction through facial stimuli in children with high-functioning ASD., This work was partly sponsored by the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project) and by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI) project Musical AI—PID2019-111403GB-I00/AEI/10.13039/501100011033.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A deep-learning based framework for source separation, analysis, and synthesis of choral ensembles

  • Chandna, Pritish
  • Cuesta, Helena
  • Petermann, Darius
  • Gómez Gutiérrez, Emilia, 1975-
Choral singing in the soprano, alto, tenor and bass (SATB) format is a widely practiced and studied art form with significant cultural importance. Despite the popularity of the choral setting, it has received little attention in the field of Music Information Retrieval. However, the recent publication of high-quality choral singing datasets as well as recent developments in deep learning based methodologies applied to the field of music and speech processing, have opened new avenues for research in this field. In this paper, we use some of the publicly available choral singing datasets to train and evaluate state-of-the-art source separation algorithms from the speech and music domains for the case of choral singing. Furthermore, we evaluate existing monophonic F0 estimators on the separated unison stems and propose an approximation of the perceived F0 of a unison signal. Additionally, we present a set of applications combining the proposed methodologies, including synthesizing a single singer voice from the unison, and transposing and remixing the separated stems into a synthetic multi-singer choral signal. We finally conduct a set of listening tests to perform a perceptual evaluation of the results we obtain with the proposed methodologies., This work is partially supported by the European Commission under the TROMPA project (H2020 770376), and the project Musical AI (PID 2019-111403GB-I00/AEI/10.13039/501100011033) funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Error monitoring and correction in violin performance: an EEG study

  • Blanco, Angel David
  • Costa-Faidella, Jordi
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Dalmazzo, David
  • Ramirez Melendez, Rafael
  • San Miguel Insúa, Iria
Previous research using electroencephalography (EEG) has investigated neural error-related music production processes in expert pianists reporting a frontal event-related negativity followed by a P300 event-related potential (ERP) after mistuned notes. However, piano playing does not rely on auditory feedback and corrective movements when dealing with accurate note tuning, thus offering an incomplete picture about error monitoring processes in players of many other musical instruments, including the human voice. We here designed a setup allowing us to collect the EEG of expert bow-string players while performing short melodies on the violin, (a fretless instrument) while also manipulating in real-time the produced pitch in a controlled manner. 15 expert bow-string players were asked to 1) listen to a reference melody of four notes; 2) play that melody with a violin (Active Condition); and 3) listen passively to the replay of their performance (Replayed Condition). Randomly, the auditory feedback of one of the four melody notes was manipulated in the Active Condition by lowering or lifting the pitch by half semitone. We found that mistuned notes, independently of whether they were manipulated externally or were produced by natural playing errors, elicited an f-ERN followed by a P300. We also found that the amplitude of the f-ERN was larger for faster corrective movements than for slower ones. In addition, between the f-ERN and the P300, we also identified two negative components-one in parietal areas and another in central-right regions-whose amplitudes were greater for slower corrective movements. We suggest the existence of two error-monitoring systems: a "fast" system mediated by the medial frontal cortex that quickly corrects errors when certainty is high and a "slow" system mediated by the posterior parietal cortex that accumulates evidence under uncertainty before triggering correction., This work was partly sponsored by the Musical AI project (PID2019-111403GB-I00) funded by MCIN/AEI/ 10.13039/501100011033 and by 'ERDF A way of making Europe'; by the Project PID2021-128790NB-I00, funded by MCIN/AEI/ 10.13039/501100011033 / FEDER UE; and by the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project). It has been additionally supported by the Excellence Research Group 2017SGR-974 funded by the Secretaria d'Universitats i Recerca del Departament d'Empresa i Coneixement de la Generalitat de Catalunya and by the MDM-2017-0729-18-2M Maria de Maeztu Center of Excellence UBNeuro, funded by MCIN/AEI/ 10.13039/501100011033. ISM was supported by grant RYC-2013-12577, funded by MCIN/AEI/ 10.13039/501100011033 and by 'ESF Investing in your future'. Open access publication fees were partly covered by the University of Barcelona (Ajudes de la Universitat de Barcelona per a publicar en accés obert).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Voice assignment in vocal quartets using deep learning models based on pitch salience

  • Cuesta, Helena
  • Gómez Gutiérrez, Emilia, 1975-
This paper deals with the automatic transcription of four-part, a cappella singing, audio performances. In particular, we exploit an existing, deep-learning based, multiple F0 estimation method and complement it with two neural network architectures for voice assignment (VA) in order to create a music transcription system that converts an input audio mixture into four pitch contours. To train our VA models, we create a novel synthetic dataset by collecting 5381 choral music scores from public-domain music archives, which we make publicly available for further research. We compare the performance of the proposed VA models on different types of input data, as well as to a hidden Markov model-based baseline system. In addition, we assess the generalization capabilities of these models on audio recordings with differing pitch distributions and vocal music styles. Our experiments show that the two proposed models, a CNN and a ConvLSTM, have very similar performance, and both of them outperform the baseline HMM-based system. We also observe a high confusion rate between the alto and tenor voice parts, which commonly have overlapping pitch ranges, while the bass voice has the highest scores in all evaluated scenarios., This work is partially supported by the European Commission under the TROMPA project (H2020 770376), the Spanish Ministry of Science and Innovation under the Musical AI project (PID2019-111403GB-I00), and by AGAUR (Generalitat de Catalunya) through an FI Predoctoral Grant (2018FI-B01015).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Applications of biological and physiological signals in commercial video gaming and game research: a review

  • Hughes, Alayna
  • Jordà Puig, Sergi
Video gaming is now available as a fully immersive experience that creates responsive inputs and outputs concerning the user, and some experimental developers have integrated the use of the voice, brain, or muscles as input controls. The use of physiological signal equipment can provide valuable information regarding the emotion of a player or patient during gameplay. In this article, we discuss five of the most common biosignals that are used in gaming research, and their function and devices that may be used for measurement. We break down those individual signals and present examples of research studies that implement them. We also discuss the usage of biological signals within commercial gaming and conclude with some possible future directions for the use of biological signals in gaming and game research., This research has been partially supported by the project Musical AI - PID2019-111403GB-I00/AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).