Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 77
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Multiple F0 estimation in vocal ensembles using convolutional neural networks

  • Cuesta, Helena
  • McFee, Brian
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., This paper addresses the extraction of multiple F0 values from polyphonic and a cappella vocal performances using convolutional neural networks (CNNs). We address the major challenges of ensemble singing, i.e., all melodic sources are vocals and singers sing in harmony. We build upon an existing architecture to produce a pitch salience function of the input signal, where the harmonic constantQ transform (HCQT) and its associated phase differentials are used as an input representation. The pitch salience function is subsequently thresholded to obtain a multiple F0 estimation output. For training, we build a dataset that comprises several multi-track datasets of vocal quartets with F0 annotations. This work proposes and evaluates a set of CNNs for this task in diverse scenarios and data configurations, including recordings with additional reverb. Our models outperform a state-of-the-art method intended for the same music genre when evaluated with an increased F0 resolution, as well as a general-purpose method for multi-F0 estimation. We conclude with a discussion on future research directions., The authors would like to thank Rodrigo Schramm and Emmanouil Benetos for sharing the BSQ and BC datasets for this research. Helena Cuesta is supported by the FI Predoctoral Grant from AGAUR (Generalitat de Catalunya). This work is partially supported by the European Commission under the TROMPA project (H2020 770376) and MARL-NYU (as part of a two-months research stay).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Interweaving and enriching digital music collections for scholarship, performance, and enjoyment

  • Weigl, David M.
  • Goebl, Werner
  • Crawford, Tim
  • Gkiokas, Aggelos
  • Gutiérrez Páez, Nicolás Felipe
  • Porter, Alastair
  • Santos Rodríguez, Patrícia
  • Karreman, Casper
  • Vroomen, Ingmar
  • Liem, Cynthia C. S.
  • Sarasúa Berodia, Álvaro
  • van Tilburg, Marcel
Comunicació presentada a: 6th International Conference on Digital Libraries for Musicology (DLfM) celebrat el 9 de novembre de 2019 a La Haia, Països Baixos., The turn toward the digital has opened up previously difficult to access musical materials to wider musicological scholarship. Digital repositories provide access to publicly licensed score images, score encodings, textual resources, audiovisual recordings, and music metadata. While each repository reveals rich information for scholarly investigation, the unified exploration and analysis of separate digital collections remains a challenge. TROMPA—Towards Richer Online Music Public-domain Archives—addresses this through a knowledge graph interweaving composers, performers, and works described in established digital music libraries, facilitating discovery and combined access of complementary materials across collections. TROMPA provides for contribution of expert insights as citable, provenanced annotations, supporting analytical workflows and scholarly communication. Beyond scholars, the project targets four further user types: instrumental players; choir singers; orchestras; and music enthusiasts; with corresponding web applications providing specialised views of the same underlying knowledge graph. Thus, scholars’ annotations provide contextual information to other types of users; while performers’ rehearsal recordings and performative annotations, conductors’ marked up scores, and enthusiasts’ social discussions and listening behaviours, become available to scholarly analysis (per user consent). The knowledge graph is exposed as Linked Data, adhering to the FAIR principles of making data Findable, Accessible, Interoperable, and Re-usable, and supporting further interlinking, re-interpretation and re-use beyond the immediate scope of the project., This project has received funding from the European Union’s Horizon 2020 research and innovation programme H2020-EU.3.6.3.1. - Study European heritage, memory, identity, integration and cultural interaction and translation, including its representations in cultural and scientific collections, archives and museums, to better inform and understand the present by richer interpretations of the past under grant agreement No. 770376.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

FAIR interconnection and enrichment of public-domain music resources on the Web

  • Weigl, David M.
  • Crawford, Tim
  • Gkiokas, Aggelos
  • Goebl, Werner
  • Gómez Gutiérrez, Emilia, 1975-
  • Gutiérrez Páez, Nicolás Felipe
  • Liem, Cynthia C. S.
  • Santos Rodríguez, Patrícia
Vast amounts of publicly licensed classical music resources are housed within many different repositories on the Web encompassing richly diverse facets of information—including bibliographical and biographical data, digitized images of music notation, music score encodings, audiovisual performance recordings, derived feature data, scholarly commentaries, and listener reactions. While these varied perspectives ought to contribute to greater holistic understanding of the music objects under consideration, in practice, such repositories are typically minimally connected. The TROMPA project aims to improve this situation by interconnecting and enriching public-domain music repositories. This is achieved, on the one hand, by the application of automated, cutting-edge Music Information Retrieval techniques, and on the other, by the development of contribution mechanisms enabling users to integrate their expertise. Information within established repositories is interrelated with data generated by the project within a data infrastructure whose design is guided by the FAIR principles of data management and stewardship: making music information Findable, Accessible, Interoperable, and Reusable. We provide an overview of challenges of description, identification, representation, contribution, and reliability toward applying the FAIR principles to music information, and outline TROMPA’s implementational approach to overcoming these challenges. This approach applies a graph-based data infrastructure to interrelate information hosted in different repositories on the Web within a unifying data model (a ‘knowledge graph’). Connections are generated across different representations of music content beyond the catalogue level, for instance connecting note elements within score encodings to corresponding moments in performance time-lines. Contributions of user data are supported via privacy-first mechanisms that retain control of such data with the contributing user. Provenance information is captured throughout, supporting reproducibility and re-use of the data both within and outside the context of the project., The TROMPA Project is funded under the European Union’s Horizon 2020 research and innovation programme H2020-EU.3.6.3.1. - Study European heritage, memory, identity, integration and cultural interaction and translation, including its representations in cultural and scientific collections, archives and museums, to better inform and understand the present by richer interpretations of the past under grant agreement No 770376.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

MediaEval 2018 AcousticBrainz genre task: A baseline combining deep feature embeddings across datasets

  • Oramas, Sergio
  • Bogdanov, Dmitry
  • Porter, Alastair
Comunicació presentada al MediaEval 2018 Workshop celebrat a Sophia Antipolis (França) del 29 al 31 d'octubre de 2018., In this paper we present a baseline approach for the MediaEval 2018 AcousticBrainz Genre Task that takes advantage of stacking multiple feature embeddings learned on individual genre datasets by simple deep learning architectures. Although we employ basic neural networks, the combination of their deep feature embeddings provides a significant gain in performance compared to each individual network., This research has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreements No 688382 (AudioCommons) and 770376-2 (TROMPA), as well as the Ministry of Economy and Competitiveness of the Spanish Government (Reference: TIN2015-69935-P).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Emotion annotation of music: a citizen science approach

  • Gutiérrez Páez, Nicolás Felipe
  • Gómez Cañón, Juan Sebastián
  • Porcaro, Lorenzo
  • Santos Rodríguez, Patrícia
  • Hernández Leo, Davinia
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: 27th International Conference, CollabTech 2021 celebrat de manera virtual del 31 d'agost al 3 de setembre de 2021., The understanding of the emotions in music has motivated research across diverse areas of knowledge for decades. In the field of computer science, there is a particular interest in developing algorithms to “predict” the emotions in music perceived by or induced to a listener. However, the gathering of reliable “ground truth” data for modeling the emotional content of music poses challenges, since tasks related with annotations of emotions are time consuming, expensive and cognitively demanding due to its inherent subjectivity and its cross-disciplinary nature. Citizen science projects have proven to be a useful approach to solve these types of problems where there is a need for recruiting collaborators for massive scale tasks. We developed a platform for annotating emotional content in musical pieces following a citizen science approach, to benefit not only the researchers, who benefit from the generated dataset, but also the volunteers, who are engaged to collaborate on the research project, not only by providing annotations but also through their self and community-awareness about the emotional perception of the music. Likewise, gamification mechanisms motivate the participants to explore and discover new music based on the emotional content. Preliminary user evaluations showed that the platform design is in line with the motivations of the general public, and that the citizen science approach offers an iterative refinement to enhance the quantity and quality of contributions by involving volunteers in the design process. The usability of the platform was acceptable, although some of the features require improvements., This work has been partially funded by the TROMPA project, European Union’s Horizon 2020 research and innovation programme under grant agreement no. 770376. TIDE-UPF also acknowledges the support by FEDER, the National Research Agency of the Spanish Ministry of Science and Innovation, TIN2017-85179-C3-3-R, PID2020-112584RB-C33, MDM-2015-0502, the Ramon y Cajal programme (P. Santos) and by ICREA under the ICREA Academia programme (D. Hernández-Leo, Serra Hunter).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

End-to-end sound source separation conditioned on instrument labels

  • Slizovskaia, Olga
  • Kim, Leo
  • Haro Ortega, Gloria
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing celebrat de 12 al 17 de maig de 2019 a Brighton, Regne Unit., Can we perform an end-to-end music source separation with a variable number of sources using a deep learning model? This paper presents an extension of the Wave-UNet [1] model which allows end-to-end monaural source separation with a non-fixed number of sources. Furthermore, we propose multiplicative conditioning with instrument labels at the bottleneck of the Wave-U-Net and show its effect on the separation results. This approach can be further extended to other types of conditioning such as audio-visual source separation and score-informed source separation., This work has received funding from the Maria de Maeztu Programme (MDM-2015-0502), ERC Innovation Programme (grant 770376, TROMPA), and MINECO/FEDER UE project (TIN2015-70410-C2-1-R).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Deep learning based source separation applied to choir ensembles

  • Petermann, Darius
  • Chandna, Pritish
  • Cuesta, Helena
  • Bonada, Jordi, 1973-
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Choral singing is a widely practiced form of ensemble singing wherein a group of people sing simultaneously in polyphonic harmony. The most commonly practiced setting for choir ensembles consists of four parts; Soprano, Alto, Tenor and Bass (SATB), each with its own range of fundamental frequencies (F0s). The task of source separation for this choral setting entails separating the SATB mixture into the constituent parts. Source separation for musical mixtures is well studied and many deep learning based methodologies have been proposed for the same. However, most of the research has been focused on a typical case which consists in separating vocal, percussion and bass sources from a mixture, each of which has a distinct spectral structure. In contrast, the simultaneous and harmonic nature of ensemble singing leads to high structural similarity and overlap between the spectral components of the sources in a choral mixture, making source separation for choirs a harder task than the typical case. This, along with the lack of an appropriate consolidated dataset has led to a dearth of research in the field so far. In this paper we first assess how well some of the recently developed methodologies for musical source separation perform for the case of SATB choirs. We then propose a novel domain-specific adaptation for conditioning the recently proposed U-Net architecture for musical source separation using the fundamental frequency contour of each of the singing groups and demonstrate that our proposed approach surpasses results from domain-agnostic architectures., The TITANX used for this research was donated by the NVIDIA Corporation. This work is partially supported by the Towards Richer Online Music Public-domain Archives (TROMPA H2020 770376) project. Helena Cuesta is supported by the FI Predoctoral Grant from AGAUR (Generalitat de Catalunya). The authors would like to thank Rodrigo Schramm and Emmanouil Benetos for sharing their singing voice datasets for this research.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Accurate and scalable version identification using musically-motivated embeddings

  • Yesiler, Furkan
  • Serrà Julià, Joan
  • Gómez Gutiérrez, Emilia, 1975-
The version identification (VI) task deals with the automatic detection of recordings that correspond to the same underlying musical piece. Despite many efforts, VI is still an open problem, with much room for improvement, specially with regard to combining accuracy and scalability. In this paper, we present MOVE, a musically-motivated method for accurate and scalable version identification. MOVE achieves state-of-the-art performance on two publicly-available benchmark sets by learning scalable embeddings in an Euclidean distance space, using a triplet loss and a hard triplet mining strategy. It improves over previous work by employing an alternative input representation, and introducing a novel technique for temporal content summarization, a standardized latent space, and a data augmentation strategy specifically designed for VI. In addition to the main results, we perform an ablation study to highlight the importance of our design choices, and study the relation between embedding dimensionality and model performance., Comunicació presentada a: ICASSP 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, celebrat en línia del 4 al 8 de maig de 2020., This work is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skodowska-Curie grant agreement No. 765068, and by TROMPA, the Horizon 2020 project 770376-2.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

The MediaEval 2018 AcousticBrainz genre task: content-based music genre recognition from multiple sources

  • Bogdanov, Dmitry
  • Porter, Alastair
  • Urbano, Julián
  • Schreiber, Hendrik
Comunicació presentada al MediaEval 2018 Workshop celebrat a Sophia Antipolis (França) del 29 al 31 d'octubre de 2018., This paper provides an overview of the AcousticBrainz Genre Task organized as part of the MediaEval 2018 Benchmarking Initiative for Multimedia Evaluation. The task is focused on content-based music genre recognition using genre annotations from multiple sources and large-scale music features data available in the AcousticBrainz database. The goal of our task is to explore how the same music pieces can be annotated differently by different communities following different genre taxonomies, and how this should be addressed by content-based genre recognition systems. We present the task challenges, the employed ground-truth information and datasets, and the evaluation methodology., We thank all contributors to AcousticBrainz. This research has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreements No 688382 (AudioCommons) and 770376-2 (TROMPA), as well as the Ministry of Economy and Competitiveness of the Spanish Government (Reference: TIN2015-69935-P). We also thank tagtraum industries for providing the Tagtraum genre annotations.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A deep learning based analysis-synthesis framework for unison singing

  • Chandna, Pritish
  • Cuesta, Helena
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Unison singing is the name given to an ensemble of singers simultaneously singing the same melody and lyrics. While each individual singer in a unison sings the same principle melody, there are slight timing and pitch deviations between the singers, which, along with the ensemble of timbres, give the listener a perceived sense of "unison". In this paper, we present a study of unison singing in the context of choirs; utilising some recently proposed deep-learning based methodologies, we analyse the fundamental frequency (F0) distribution of the individual singers in recordings of unison mixtures. Based on the analysis, we propose a system for synthesising a unison signal from an a cappella input and a single voice prototype representative of a unison mixture. We use subjective listening test to evaluate perceptual factors of our proposed system for synthesis, including quality, adherence to the melody as well the degree of perceived unison., The TITANX used for this research was donated by the NVIDIA Corporation. This work is partially supported by the Towards Richer Online Music Public-domain Archives (TROMPA H2020 770376) project. Helena Cuesta is supported by the FI Predoctoral Grant from AGAUR (Generalitat de Catalunya).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Voice assignment in vocal quartets using deep learning models based on pitch salience

  • Cuesta, Helena
  • Gómez Gutiérrez, Emilia, 1975-
This paper deals with the automatic transcription of four-part, a cappella singing, audio performances. In particular, we exploit an existing, deep-learning based, multiple F0 estimation method and complement it with two neural network architectures for voice assignment (VA) in order to create a music transcription system that converts an input audio mixture into four pitch contours. To train our VA models, we create a novel synthetic dataset by collecting 5381 choral music scores from public-domain music archives, which we make publicly available for further research. We compare the performance of the proposed VA models on different types of input data, as well as to a hidden Markov model-based baseline system. In addition, we assess the generalization capabilities of these models on audio recordings with differing pitch distributions and vocal music styles. Our experiments show that the two proposed models, a CNN and a ConvLSTM, have very similar performance, and both of them outperform the baseline HMM-based system. We also observe a high confusion rate between the alto and tenor voice parts, which commonly have overlapping pitch ranges, while the bass voice has the highest scores in all evaluated scenarios., This work is partially supported by the European Commission under the TROMPA project (H2020 770376), the Spanish Ministry of Science and Innovation under the Musical AI project (PID2019-111403GB-I00), and by AGAUR (Generalitat de Catalunya) through an FI Predoctoral Grant (2018FI-B01015).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

WGansing: a multi-voice singing voice synthesizer based on the Wasserstein-Gan

  • Chandna, Pritish
  • Blaauw, Merlijn
  • Bonada, Jordi, 1973-
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada al EUSIPCO 2019: 27th European Signal Processing Conference, celebrat del 2 al 6 de setembre de 2019 a La Corunya, Espanya., We present a deep neural network based singing voice synthesizer, inspired by the Deep Convolutions Generative Adversarial Networks (DCGAN) architecture and optimized using the Wasserstein-GAN algorithm. We use vocoder parameters for acoustic modelling, to separate the influence of pitch and timbre. This facilitates the modelling of the large variability of pitch in the singing voice. Our network takes a block of consecutive frame-wise linguistic and fundamental frequency features, along with global singer identity as input and outputs vocoder features, corresponding to the block of features. This block-wise approach, along with the training methodology allows us to model temporal dependencies within the features of the input block. For inference, sequential blocks are concatenated using an overlap-add procedure. We show that the performance of our model is competitive with regards to the state-of-the-art and the original sample using objective metrics and a subjective listening test. We also present examples of the synthesis on a supplementary website and the source code via GitHub., This work is partially supported by the European Commission under the TROMPA project (H2020 770376). The TITAN X used for this research was donated by the NVIDIA Corporation.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Diversity by design in music recommender systems

  • Porcaro, Lorenzo
  • Castillo, Carlos
  • Gómez Gutiérrez, Emilia, 1975-
Music Recommender Systems (Music RS) are nowadays pivotal in shaping the listening experience of people all around the world. Partly driven by the commercial application of this technology, music recommendation research has gained increasing attention both within and outside the Music Information Retrieval (MIR) community. Thanks also to the widespread use of recommender systems in music streaming services, it has been possible to enhance several characteristics of such systems in terms of performance, design, and user experience. Nonetheless, imagining Music RS only from an application-driven perspective may generate an incomplete view of how this technology is affecting people’s habitus, from the decision-making processes to the formation of musical taste and opinions. In this overview, we address the concept of diversity in music recommendation, and taking a value-driven approach we review diversity-related methodologies proposed in the Music RS literature. Additionally, by taking as an example the wider context of Information Technology (IT), we present the elements interacting in the diversity by design paradigm. We do that to acknowledge the lack of a comprehensive framework in Music RS research to address diversity, until now mostly driven by empirical results and fragmented in different application areas. Maintaining an interdisciplinary perspective, we discuss some challenges that MIR practitioners may face when researching Music RS, going beyond the search for better performance and instead questioning the theoretical foundations on which to base future research., This work is partially supported by the European Commission under the TROMPA project (H2020 – grant agreement No. 770376). This work is also partially supported by the HUMAINT programme (Human Behaviour and Machine Intelligence), Joint Research Centre, European Commission. The project leading to these results received funding from “la Caixa” Foundation (ID 100010434), under the agreement LCF/PR/PR16/51110009.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Combining musical features for cover detection

  • Doras, Guillaume
  • Yesiler, Furkan
  • Serrà Julià, Joan
  • Gómez Gutiérrez, Emilia, 1975-
  • Peeters, Geoffroy
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Recent works have addressed the automatic cover detection problem from a metric learning perspective. They employ different input representations, aiming to exploit melodic or harmonic characteristics of songs and yield promising performances. In this work, we propose a comparative study of these different representations and show that systems combining melodic and harmonic features drastically outperform those relying on a single input representation. We illustrate how these features complement each other with both quantitative and qualitative analyses. We finally investigate various fusion schemes and propose methods yielding state-of-the-art performances on two publicly-available large datasets., FY is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068, and EG by TROMPA, the Horizon 2020 project 770376-2.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

How diverse is the ACII community? Analysing gender, geographical and business diversity of affective computing research

  • Hupont, Isabelle
  • Tolan, Songül
  • Freire, Ana
  • Porcaro, Lorenzo
  • Estevez, Sara
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII), celebrat del 28 de setembre a l'1 d'octubre de 2021 de manera virtual., ACII is the premier international forum for presenting the latest research on affective computing. In this work, we monitor, quantify and reflect on the diversity in ACII conference across time by computing a set of indexes. We measure diversity in terms of gender, geographic location and academia vs research centres vs industry, and consider three different actors: authors, keynote speakers and organizers. Results raise awareness on the limited diversity in the field, in all studied facets, and compared to other AI conferences. While gender diversity is relatively high, equality is far from being reached. The community is dominated by European, Asian and North American researchers, leading the rest of continents under-represented. There is also a strong absence of companies and research centres focusing on applied research and products. This study fosters discussion in the community on the need for diversity and related challenges in terms of minimizing potential biases of the developed systems to the represented groups. We intend our paper to contribute with a first analysis to consider as a monitoring tool when implementing diversity initiatives. The data collected for this study are publicly released through the European divinAI initiative., This work is partially supported by the European Commission under the TROMPA project (H2020 770376) and the HUMAINT project of the Joint Research Centre.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Exploring artist gender bias in music recommendation

  • Shakespeare, Dougal
  • Porcaro, Lorenzo
  • Gómez Gutiérrez, Emilia, 1975-
  • Castillo, Carlos
Comunicació presentada a: Workshop on the Impact of Recommender Systems, ACM RecSys 2020 celebrat de manera virtual el 25 de setembre de 2020., Music Recommender Systems (mRS) are designed to give personalised and meaningful recommendations of items (i.e. songs, playlists or artists) to a user base, thereby reflecting and further complementing individual users’ specific music preferences. Whilst accuracy metrics have been widely applied to evaluate recommendations in mRS literature, evaluating a user’s item utility from other impactoriented perspectives, including their potential for discrimination, is still a novel evaluation practice in the music domain. In this work, we center our attention on a specific phenomenon for which we want to estimate if mRS may exacerbate its impact: gender bias. Our work presents an exploratory study, analyzing the extent to which commonly deployed state of the art Collaborative Filtering (CF) algorithms may act to further increase or decrease artist gender bias. To assess group biases introduced by CF, we deploy a recently proposed metric of bias disparity on two listening event datasets: the LFM-1b dataset, and the earlier constructed Celma’s dataset. Our work traces the causes of disparity to variations in input gender distributions and user-item preferences, highlighting the effect such configurations can have on user’s gender bias after recommendation generation., This work is partially supported by the European Commission under the TROMPA project (H2020 770376).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Content based singing voice extraction from a musical mixture

  • Chandna, Pritish
  • Blaauw, Merlijn
  • Bonada, Jordi, 1973-
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: ICASSP 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, celebrat en línia del 4 al 8 de maig de 2020., We present a deep learning based methodology for extracting the singing voice signal from a musical mixture based on the underlying linguistic content. Our model follows an encoder-decoder architecture and takes as input the magnitude component of the spectrogram of a musical mixture with vocals. The encoder part of the model is trained via knowledge distillation using a teacher network to learn a content embedding, which is decoded to generate the corresponding vocoder features. Using this methodology, we are able to extract the unprocessed raw vocal signal from the mixture even for a processed mixture dataset with singers not seen during training. While the nature of our system makes it incongruous with traditional objective evaluation metrics, we use subjective evaluation via listening tests to compare the methodology to state-of-the-art deep learning based source separation algorithms. We also provide sound examples and source code for reproducibility., This work is partially supported by the Towards Richer Online Music Public-domain Archives (TROMPA H2020 770376) project.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Personalisation and profiling using algorithms and not-so-popular Colombian music: goal-directed mechanisms in music emotion recognition

  • Gómez-Cañon, Juan Sebastián
  • Lennie, Thomas Magnus
  • Eerola, Tuomas
  • Aragón, Pablo
  • Cano, Estefanía
  • Herrera, Perfecto
  • Gómez, Emilia
This work investigates how personalised Music Emotion Recognition (MER) systems may lead to sensitive profiling when applied to musically induced emotions in politically charged contexts. We focus on traditional Colombian music with explicit political content, including (1) vallenatos and social songs aligned with the left-wing guerrilla Fuerzas Armadas Revolucionarias de Colombia (FARC), and (2) corridos linked to sympathisers of the right-wing paramilitary group Autodefensas Unidas de Colombia (AUC). Using data from 49 participants with diverse political leanings, we train personalised machine learning models to predict induced emotional responses -particularly negative emotions. Our findings reveal that political identity plays a significant role in shaping emotional experiences of music with explicit political content, and that emotion recognition models can capture this variation to a certain extent. These results raise critical concerns about the potential misuse of emotion recognition technologies. What is often framed as a tool for wellbeing and emotional regulation could, in politically sensitive contexts, be repurposed for user profiling. This work highlights the ethical risks of deploying AI-driven emotion analysis without safeguards, particularly among populations that are politically or socially vulnerable. We argue that subjective emotional responses may constitute sensitive personal data, and that failing to account for their sociopolitical context could amplify harm and exclusion., JSGC's contribution was partially supported by the European Commission through the TROMPA project (H2020 770376) and the project Musical AI (PID2019-111403GB-I00/AEI/10.13039/501100011033), funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI). TML was supported by the Arts and Humanities Research Council (AHRC) through a Northern Bridge Doctoral Training Partnership scholarship (grant code: AH/L503927/1).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Less is more: faster and better music version identification with embedding distillation

  • Serrà Julià, Joan
  • Yesiler, Furkan
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Version identification systems aim to detect different renditions of the same underlying musical composition (loosely called cover songs). By learning to encode entire recordings into plain vector embeddings, recent systems have made significant progress in bridging the gap between accuracy and scalability, which has been a key challenge for nearly two decades. In this work, we propose to further narrow this gap by employing a set of data distillation techniques that reduce the embedding dimensionality of a pre-trained state-of-the-art model. We compare a wide range of techniques and propose new ones, from classical dimensionality reduction to more sophisticated distillation schemes. With those, we obtain 99% smaller embeddings that, moreover, yield up to a 3% accuracy increase. Such small embeddings can have an important impact in retrieval time, up to the point of making a real-world system practical on a standalone laptop., This work is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068, and by TROMPA, the Horizon 2020 project 770376-2.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Music recommendation diversity: a tentative framework and preliminary results

  • Porcaro, Lorenzo
  • Castillo, Carlos
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada al 1st Workshop on Designing Human-Centric MIR Systems celebrat el 2 de novembre de 2019 a Delft, Països Baixos, en el marc del 20th annual conference of the International Society for Music Information Retrieval (ISMIR)., Music recommendations are increasingly part of the listening experience of people all over the world, especially in the context of streaming services. In this scenario, recommender systems’ role is to help users in finding music that can fit their interests and tastes. However, Westerncentric perspectives in systems’ design are often subject to criticisms because of their power of reinforcing already existing cultural bias and therefore potentially impacting negatively on the music distribution mechanisms. In our research proposal, we aim to address the problem of assessing the impact of music recommendation diversity, or the lack thereof. This requires 1) the formalization of a working definition of diversity in the music field 2) the development of evaluation practices for estimating diversity in the context of music recommender systems 3) the observation of emerging impact due to music recommendation diversity 4) the proposal of countermeasures for mitigating negative or reinforcing positive impact observed. Basing on already known consequences of information technologies in political, economic and social areas, our goal is to understand the cultural impact that music recommender systems can have on our society., This work is partially supported by the European Commission under the TROMPA project (H2020 770376).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

A deep-learning based framework for source separation, analysis, and synthesis of choral ensembles

  • Chandna, Pritish
  • Cuesta, Helena
  • Petermann, Darius
  • Gómez Gutiérrez, Emilia, 1975-
Choral singing in the soprano, alto, tenor and bass (SATB) format is a widely practiced and studied art form with significant cultural importance. Despite the popularity of the choral setting, it has received little attention in the field of Music Information Retrieval. However, the recent publication of high-quality choral singing datasets as well as recent developments in deep learning based methodologies applied to the field of music and speech processing, have opened new avenues for research in this field. In this paper, we use some of the publicly available choral singing datasets to train and evaluate state-of-the-art source separation algorithms from the speech and music domains for the case of choral singing. Furthermore, we evaluate existing monophonic F0 estimators on the separated unison stems and propose an approximation of the perceived F0 of a unison signal. Additionally, we present a set of applications combining the proposed methodologies, including synthesizing a single singer voice from the unison, and transposing and remixing the separated stems into a synthetic multi-singer choral signal. We finally conduct a set of listening tests to perform a perceptual evaluation of the results we obtain with the proposed methodologies., This work is partially supported by the European Commission under the TROMPA project (H2020 770376), and the project Musical AI (PID 2019-111403GB-I00/AEI/10.13039/501100011033) funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A Framework for multi-f0 modeling in SATB choir recordings

  • Cuesta, Helena
  • Gómez Gutiérrez, Emilia, 1975-
  • Chandna, Pritish
Comunicació presentada a: Sound & Music Computing Conference celebrada del 28 al 31 de maig de 2019 a Màlaga, Espanya., Fundamental frequency (f0) modeling is an important but relatively unexplored aspect of choir singing. Performance evaluation as well as auditory analysis of singing, whether individually or in a choir, often depend on extracting f0 contours for the singing voice. However, due to the large number of singers, singing at a similar frequency range, extracting the exact individual pitch contours from choir recordings is a challenging task. In this paper, we address this task and develop a methodology for modeling pitch contours of SATB choir recordings. A typical SATB choir consists of four parts, each covering a distinct range of pitches and often with multiple singers each. We first evaluate some state-of-the-art multi-f0 estimation systems for the particular case of choirs with a single singer per part, and observe that the pitch of individual singers can be estimated to a relatively high degree of accuracy. We observe, however, that the scenario of multiple singers for each choir part (i.e. unison singing) is far more challenging. In this work we propose a methodology based on combining a multi-f0 estimation methodology based on deep learning followed by a set of traditional DSP techniques to model f0 and its dispersion instead of a single f0 trajectory for each choir part. We present and discuss our observations and test our framework with different singer configurations., This work is partially supported by the European Commission under the TROMPA project (H2020 770376) and by Spanish Ministry of Economy and Competitiveness under the CASAS project (TIN2015-70816-R). First author is supported by FI Predoctoral Grant from AGAUR (Generalitat de Catalunya).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Music emotion recognition: toward new, robust standards in personalized and context-sensitive applications

  • Gómez Cañón, Juan Sebastián
  • Cano, Estefanía
  • Eerola, Tuomas
  • Herrera Boyer, Perfecto, 1964-
  • Hu, Xiao
  • Yang, Yi-Hsuan
  • Gómez Gutiérrez, Emilia, 1975-
Emotion is one of the main reasons why people engage and interact with music [1] . Songs can express our inner feelings, produce goosebumps, bring us to tears, share an emotional state with a composer or performer, or trigger specific memories. Interest in a deeper understanding of the relationship between music and emotion has motivated researchers from various areas of knowledge for decades [2] , including computational researchers. Imagine an algorithm capable of predicting the emotions that a listener perceives in a musical piece, or one that dynamically generates music that adapts to the mood of a conversation in a film—a particularly fascinating and provocative idea. These algorithms typify music emotion recognition (MER), a computational task that attempts to automatically recognize either the emotional content in music or the emotions induced by music to the listener [3] . To do so, emotionally relevant features are extracted from music. The features are processed, evaluated, and then associated with certain emotions. MER is one of the most challenging high-level music description problems in music information retrieval (MIR), an interdisciplinary research field that focuses on the development of computational systems to help humans better understand music collections. MIR integrates concepts and methodologies from several disciplines, including music theory, music psychology, neuroscience, signal processing, and machine learning., Juan Gómez-Cañón is partially supported by the European Commission under the TROMPA project (H2020 770376).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Diversity in the music listening experience: insights from focus group interviews

  • Porcaro, Lorenzo
  • Gómez Gutiérrez, Emilia, 1975-
  • Castillo, Carlos
Comunicació presentada a: CHIIR '22: ACM SIGIR Conference on Human Information Interaction and Retrieval, celebrat del 14 al 18 de març de 2022 a Regensburg, Alemanya., Music listening in today’s digital spaces is highly characterized by the availability of huge music catalogues, accessible by people all over the world. In this scenario, recommender systems are designed to guide listeners in finding tracks and artists that best fit their requests, having therefore the power to influence the diversity of the music they listen to. Albeit several works have proposed new techniques for developing diversity-aware recommendations, little is known about how people perceive diversity while interacting with music recommendations. In this study, we interview several listeners about the role that diversity plays in their listening experience, trying to get a better understanding of how they interact with music recommendations. We recruit the listeners among the participants of a previous quantitative study, where they were confronted with the notion of diversity when asked to identify, from a series of electronic music lists, the most diverse ones according to their beliefs. As a follow-up, in this qualitative study we carry out semi-structured interviews to understand how listeners may assess the diversity of a music list and to investigate their experiences with music recommendation diversity. We report here our main findings on 1) what can influence the diversity assessment of tracks and artists’ music lists, and 2) which factors can characterize listeners’ interaction with music recommendation diversity., This work is partially supported by the European Commission under the TROMPA project (H2020 - grant agreement No. 770376). It is also part of the project Musical AI - PID2019-111403GB-I00/AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI). This work is also partially supported by the HUMAINT programme (Human Behaviour and Machine Intelligence), Joint Research Centre, European Commission. The project leading to these results received funding from "la Caixa" Foundation (ID 100010434), under the agreement LCF/PR/PR16/51110009.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Sequence-to-sequence singing synthesis using the feed-forward transformer

  • Blaauw, Merlijn
  • Bonada, Jordi, 1973-
Comunicació presentada a: ICASSP 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, celebrat en línia del 4 al 8 de maig de 2020., We propose a sequence-to-sequence singing synthesizer, which avoids the need for training data with pre-aligned phonetic and acoustic features. Rather than the more common approach of a content-based attention mechanism combined with an autoregressive decoder, we use a different mechanism suitable for feed-forward synthesis. Given that phonetic timings in singing are highly constrained by the musical score, we derive an approximate initial alignment with the help of a simple duration model. Then, using a decoder based on a feed-forward variant of the Transformer model, a series of self-attention and convolutional layers refines the result of the initial alignment to reach the target acoustic features. Advantages of this approach include faster inference and avoiding the exposure bias issues that affect autoregressive models trained by teacher forcing. We evaluate the effectiveness of this model compared to an autoregressive baseline, the importance of self-attention, and the importance of the accuracy of the duration model., This work was funded by TROMPA H2020 No 770376.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Conditioned source separation for musical instrument performances

  • Slizovskaia, Olga
  • Haro Ortega, Gloria
  • Gómez Gutiérrez, Emilia, 1975-
In music source separation, the number of sources may vary for each piece and some of the sources may belong to the same family of instruments, thus sharing timbral characteristics and making the sources more correlated. This leads to additional challenges in the source separation problem. This paper proposes a source separation method for multiple musical instruments sounding simultaneously and explores how much additional information apart from the audio stream can lift the quality of source separation. We explore conditioning techniques at different levels of a primary source separation network and utilize two extra modalities of data, namely presence or absence of instruments in the mixture, and the corresponding video stream data., This work was funded in part by ERC Innovation Programme (grant 770376, TROMPA); Spanish Ministry of Economy and Competitiveness under the Mar´ıa de Maeztu Units of Excellence Program (MDM-2015-0502) and the Social European Funds; the MICINN/FEDER UE project with reference PGC2018-098625-B-I00; and the H2020-MSCARISE-2017 project with reference 777826 NoMADS. We gratefully acknowledge NVIDIA for the donation of GPUs used for the experiments.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Da-TACOS: A dataset for cover song identification and understanding

  • Yesiler, Furkan
  • Tralie, Chris
  • Correya, Albin Andrew
  • Silva, Diego F.
  • Tovstogan, Philip
  • Gómez Gutiérrez, Emilia, 1975-
  • Serra, Xavier
Comunicació presentada a: 20th annual conference of the International Society for Music Information Retrieval (ISMIR) celebrat del 4 al 8 de novembre de 2019 a Delft, Països Baixos., This paper focuses on Cover Song Identification (CSI), an important research challenge in content-based Music Information Retrieval (MIR). Although the task itself is interesting and challenging for both academia and industry scenarios, there are a number of limitations for the advancement of current approaches. We specifically address two of them in the present study. First, the number of publicly available datasets for this task is limited, and there is no publicly available benchmark set that is widely used among researchers for comparative algorithm evaluation. Second, most of the algorithms are not publicly shared and reproducible, limiting the comparison of approaches. To overcome these limitations we propose Da-TACOS, a DaTAset for COver Song Identification and Understanding, and two frameworks for feature extraction and benchmarking to facilitate reproducibility. Da-TACOS contains 25K songs represented by unique editorial metadata plus 9 low- and mid-level features pre-computed with open source libraries, and is divided into two subsets. The Cover Analysis subset contains audio features (e.g. key, tempo) that can serve to study how musical characteristics vary for cover songs. The Benchmark subset contains the set of features that have been frequently used in CSI research, e.g. chroma, MFCC, beat onsets etc. Moreover, we provide initial benchmarking results of a selected number of state-of-the-art CSI algorithms using our dataset, and for reproducibility, we share a GitHub repository containing the feature extraction and benchmarking frameworks., This work is partially supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068, and by TROMPA, the Horizon 2020 project 770376-2.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Personalized musically induced emotions of not-so-popular Colombian music

  • Gómez Cañón, Juan Sebastián
  • Herrera Boyer, Perfecto, 1964-
  • Cano, Estefanía
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada al workshop Human Centered AI inclòs a: 35th Conference on Neural Information Processing Systems (NeurIPS 2021) celebrat el 13 de desembre de manera virtual., This work presents an initial proof of concept of how Music Emotion Recognition (MER) systems could be intentionally biased with respect to annotations of musically-induced emotions in a political context. In specific, we analyze traditional Colombian music containing politically-charged lyrics of two types: (1) vallenatos and social songs from the “left-wing” guerrilla Fuerzas Armadas Revolucionarias de Colombia (FARC) and (2) corridos from the “right-wing” paramilitaries Autodefensas Unidas de Colombia (AUC). We train personalized machine learning models to predict induced emotions for three users with diverse political views – we aim at identifying the songs that may induce negative emotions for a particular user, such as anger and fear. To this extent, a user’s emotion judgements could be interpreted as problematizing data – subjective emotional judgments could in turn be used to influence the user in a human-centered machine learning environment. In short, highly desired “emotion regulation” applications could potentially deviate to “emotion manipulation” – the recent discredit of emotion recognition technologies might transcend ethical issues of diversity and inclusion., The research work conducted at the Universitat Pompeu Fabra is partially supported by the Eu- ropean Commission under the TROMPA project (H2020 770376) and the Project Musical AI - PID2019-111403GB-I00/AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

20 years of playlists: A statistical analysis on popularity and diversity

  • Porcaro, Lorenzo
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: 20th annual conference of the International Society for Music Information Retrieval (ISMIR) celebrart del 4 al 8 de novembre de 2019 a Delft, Països Baixos., Grouping songs together, according to music preferences, mood or other characteristics, is an activity which reflects personal listening behaviours and tastes. In the last two decades, due to the increasing size of music catalogue accessible and to improvements of recommendation algorithms, people have been exposed to new ways for creating playlists. In this work, through the statistical analysis of more than 400K playlists from four datasets, created in different temporal and technological contexts, we aim to understand if it is possible to extract information about the evolution of humans strategies for playlist creation. We focus our analysis on two driving concepts of the Music Information Retrieval literature: popularity and diversity., This work is partially supported by the European Commission under the TROMPA project (H2020 770376).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A Model for evaluating popularity and semantic information variations in radio listening sessions

  • Porcaro, Lorenzo
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: 1st Workshop on the Impact of Recommender Systems dins el 13th ACM Conference on Recommender Systems (RecSys 2019) celebrat el 19 de setembre de 2019 a Copenhagen, Dinamarca., Listening to music radios is an activity that since the 20th century is part of the cultural habits for people all over the world. While in the case of analog radios DJs are in charge of selecting the music to be broadcasted, nowadays recommender systems analyzing users’ behaviours can automatically generate radios tailored to users’ musical taste. Nonetheless, in both cases listening sessions do not depend on the listener choices, but on a set of external recommendations received. In this preliminary study, we propose a model for estimating features’ variation during listening sessions, comparing different scenarios, namely analog radios, personalized and not-personalized streaming radios. In particular, we focus on the analysis of track popularity and semantic information, features well-established in the Music Information Retrieval literature. The presented model aims to quantify the possible impacts of the sessions’ variation on the user listening experience., This work is partially supported by the European Commission under the TROMPA project (H2020 770376).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Recognizing musical entities in user-generated content

  • Porcaro, Lorenzo
  • Saggion, Horacio
Comunicació presentada a: 20th International Conference on Computational Linguistics and Intelligent Text Processing (CICLing) celebrat del 7 al 13 d'abril de 2019 a La Rochelle, França., Recognizing Musical Entities is important for Music Information Retrieval (MIR) since it can improve the performance of several tasks such as music recommendation, genre classification or artist similarity. However, most entity recognition systems in the music domain have concentrated on formal texts (e.g. artists’ biographies, encyclopedic articles, etc.), ignoring rich and noisy user-generated content. In this work, we present a novel method to recognize musical entities in Twitter content generated by users following a classical music radio channel. Our approach takes advantage of both formal radio schedule and users’ tweets to improve entity recognition. We instantiate several machine learning algorithms to perform entity recognition combining task-specific and corpus-based features. We also show how to improve recognition results by jointly considering formal and user-generated content., This work is partially supported by the European Commission under the TROMPA project (H2020 770376), and by the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015- 0502).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Transfer learning from speech to music: towards language-sensitive emotion recognition models

  • Gómez Cañón, Juan Sebastián
  • Cano, Estefanía
  • Herrera Boyer, Perfecto, 1964-
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: 28th European Signal Processing Conference (EUSIPCO 2020) celebrat del 18 al 22 de gener de 2021 a Amsterdam, Països Baixos., In this study, we address emotion recognition using unsupervised feature learning from speech data, and test its transferability to music. Our approach is to pre-train models using speech in English and Mandarin, and then fine-tune them with excerpts of music labeled with categories of emotion. Our initial hypothesis is that features automatically learned from speech should be transferable to music. Namely, we expect the intra-linguistic setting (e.g., pre-training on speech in English and fine-tuning on music in English) should result in improved performance over the cross-linguistic setting (e.g., pre-training on speech in English and fine-tuning on music in Mandarin). Our results confirm previous research on cross-domain transferability, and encourage research towards language-sensitive Music Emotion Recognition (MER) models., The research work conducted in the Music Technology Group at the Universitat Pompeu Fabra is partially supported by the European Commission under the TROMPA project (H2020 770376).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

TROMPA-MER: an open dataset for personalized music emotion recognition

  • Gómez Cañón, Juan Sebastián
  • Gutiérrez Páez, Nicolás Felipe
  • Porcaro, Lorenzo
  • Porter, Alastair
  • Cano, Estefanía
  • Herrera Boyer, Perfecto, 1964-
  • Gkiokas, Aggelos
  • Santos Rodríguez, Patrícia
  • Hernández Leo, Davinia
  • Karreman, Casper
  • Gómez Gutiérrez, Emilia, 1975-
We present a platform and a dataset to help research on Music Emotion Recognition (MER). We developed the Music Enthusiasts platform aiming to improve the gathering and analysis of the so-called “ground truth” needed as input to MER systems. Firstly, our platform involves engaging participants using citizen science strategies and generate music emotion annotations – the platform presents didactic information and musical recommendations as incentivization, and collects data regarding demographics, mood, and language from each participant. Participants annotated each music excerpt with single free-text emotion words (in native language), distinct forced-choice emotion categories, preference, and familiarity. Additionally, participants stated the reasons for each annotation – including those distinctive of emotion perception and emotion induction. Secondly, our dataset was created for personalized MER and contains information from 181 participants, 4721 annotations, and 1161 music excerpts. To showcase the use of the dataset, we present a methodology for personalization of MER models based on active learning. The experiments show evidence that using the judgment of the crowd as prior knowledge for active learning allows for more effective personalization of MER systems for this particular dataset. Our dataset is publicly available and we invite researchers to use it for testing MER systems., This work was funded by the European Commission under the TROMPA project (H2020 770376) and the Project Musical AI - PID2019-111403GB-I00/AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A Vocoder based method for singing voice extraction

  • Chandna, Pritish
  • Blaauw, Merlijn
  • Bonada, Jordi, 1973-
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), celebrat del 12 al 17 de maig de 2019 a Brighton, Regne Unit., This paper presents a novel method for extracting the vocal track from a musical mixture. The musical mixture consists of a singing voice and a backing track which may comprise of various instruments. We use a convolutional network with skip and residual connections as well as dilated convolutions to estimate vocoder parameters, given the spectrogram of an input mixture. The estimated parameters are then used to synthesize the vocal track, without any interference from the backing track. We evaluate our system, through objective metrics pertinent to audio quality and interference from background sources, and via a comparative subjective evaluation. We use open-source source separation systems based on Non-negative Matrix Factorization (NMFs) and Deep Learning methods as benchmarks for our system and discuss future applications for this particular algorithm., The TITANX used for this research was donated by the NVIDIA Corporation. This work is partially supported by the Towards Richer Online Music Public-domain Archives (TROMPA) project.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Perceptions of diversity in electronic music: the impact of listener, artist, and track characteristics

  • Porcaro, Lorenzo
  • Gómez Gutiérrez, Emilia, 1975-
  • Castillo, Carlos
Shared practices to assess the diversity of retrieval system results are still debated in the Information Retrieval community, partly because of the challenges of determining what diversity means in specific scenarios, and of understanding how diversity is perceived by end-users. The field of Music Information Retrieval is not exempt from this issue. Even if fields such as Musicology or Sociology of Music have a long tradition in questioning the representation and the impact of diversity in cultural environments, such knowledge has not been yet embedded into the design and development of music technologies. In this paper, focusing on electronic music, we investigate the characteristics of listeners, artists, and tracks that are influential in the perception of diversity. Specifically, we center our attention on 1) understanding the relationship between perceived diversity and computational methods to measure diversity, and 2) analyzing how listeners' domain knowledge and familiarity influence such perceived diversity. To accomplish this, we design a user-study in which listeners are asked to compare pairs of lists of tracks and artists, and to select the most diverse list from each pair. We compare participants' ratings with results obtained through computational models built using audio tracks' features and artist attributes. We find that such models are generally aligned with participants' choices when most of them agree that one list is more diverse than the other, while they present a mixed behaviour in cases where participants have little agreement. Moreover, we observe how differences in domain knowledge, familiarity, and demographics can influence the level of agreement among listeners, and between listeners and diversity metrics computed automatically., This work is partially supported by the European Commission under the TROMPA project (H2020 - grant agreement No. 770376). This work is also partially supported by the HUMAINT programme (Human Behaviour and Machine Intelligence), Joint Research Centre, European Commission. The project leading to these results received funding from "la Caixa" Foundation (ID 100010434), under the agreement LCF/PR/PR16/51110009.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Cough-based COVID-19 detection with contextual attention convolutional neural networks and gender information

  • Mallol Ragolta, Adrià
  • Cuesta, Helena
  • Gómez Gutiérrez, Emilia, 1975-
  • Schuller, Björn
Comunicació presentada a INTERSPEECH 2021, celebrat del 30 d'agost al 3 de setembre de 2021 a Brno, Txèquia., The aim of this contribution is to automatically detect COVID19 patients by analysing the acoustic information embedded in coughs. COVID-19 affects the respiratory system, and, consequently, respiratory-related signals have the potential to contain salient information for the task at hand. We focus on analysing the spectrogram representations of cough samples with the aim to investigate whether COVID-19 alters the frequency content of these signals. Furthermore, this work also assesses the impact of gender in the automatic detection of COVID-19. To extract deep-learnt representations of the spectrograms, we compare the performance of a cough-specific, and a Resnet18 pre-trained Convolutional Neural Network (CNN). Additionally, our approach explores the use of contextual attention, so the model can learn to highlight the most relevant deep-learnt features extracted by the CNN. We conduct our experiments on the dataset released for the Cough Sound Track of the DICOVA 2021 Challenge. The best performance on the test set is obtained using the Resnet18 pre-trained CNN with contextual attention, which scored an Area Under the Curve (AUC) of 70.91 % at 80 % sensitivity., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreements No. 826506 (sustAGE) and No. 770376 (TROMPA). Further funding has been received from the FI Predoctoral Grant 2018FI-B01015 from AGAUR, Generalitat de Catalunya.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

The AcousticBrainz genre dataset: Multi-source, multi-level, multi-label, and large-scale

  • Bogdanov, Dmitry
  • Porter, Alastair
  • Schreiber, Hendrik
  • Urbano, Julián
  • Oramas, Sergio
Comunicació presentada a: 20th International Society for Music Information Retrieval Conference celebrat del 4 al 8 de novembre de 2019 a Delft, Països Baixos., This paper introduces the AcousticBrainz Genre Dataset, a large-scale collection of hierarchical multi-label genre annotations from different metadata sources. It allows researchers to explore how the same music pieces are annotated differently by different communities following their own genre taxonomies, and how this could be addressed by genre recognition systems. Genre labels for the dataset are sourced from both expert annotations and crowds, permitting comparisons between strict hierarchies and folksonomies. Music features are available via the Acoustic- Brainz database. To guide research, we suggest a concrete research task and provide a baseline as well as an evaluation method. This task may serve as an example of the development and validation of automatic annotation algorithms on complementary datasets with different taxonomies and coverage. With this dataset, we hope to contribute to developments in content-based music genre recognition as well as cross-disciplinary studies on genre metadata analysis., This research has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreements No 688382 (AudioCommons) and 770376- 2 (TROMPA), as well as the Ministry of Economy and Competitiveness of the Spanish Government (Reference: TIN2015-69935-P).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Analysis of intonation in unison choir singing

  • Cuesta, Helena
  • Gómez Gutiérrez, Emilia, 1975-
  • Martorell Domínguez, Agustín
  • Loáiciga, Felipe
Comunicació presentada a la 15th International Conference on Music Perception and Cognition celebrada del 23 al 28 de juliol a Graz, Àustria., Choir singers synchronize their fundamental frequency (f0) and timing when they perform together. In this work, we investigate several expressive characteristics of choir singing, with special emphasis on unison performances, to study how singers blend together and interact with each other by analyzing their f0 dispersion, intonation, and vibrato. We use the spectral autocorrelation algorithm to obtain the f0 trajectories of each individual singer and compute the f0 dispersion as the standard deviation of the f0 distribution. Using the Pearson correlation, we study the synchronization of singers in terms of their intonation and vibrato. Together with this study, we present a new choir singing dataset of voices singing together but recorded individually using directional close microphones. This dataset contains the individual recordings of 16 singers from a semiprofessional 4-part choir (SATB) performing 3 different songs. We found the f0 dispersion in unison sections to be in the range of 16 to 30 cents, and the largest average f0 dispersion was found in the basses. We observed that the intonation correlation varies a lot and is around 0.32 on average in all cases. The slowest piece was the one with the highest percentage of notes with vibrato, which had a frequency around 5 Hz for the whole dataset., This work is partially supported by the Spanish Ministry of Economy and Competitiveness under the CASAS project (TIN2015-70816-R) and the European Commission under the TROMPA project (H2020 770376).
RUA. Repositorio Institucional de la Universidad de Alicante
Publication Otros (Other)

Alleviating the Last Mile of Encoding: The mei-friend Package for the Atom Text Editor

  • Goebl, Werner
  • Weigl, David M.
Though MEI is widely used in music informatics and digital musicology research, the relative lack of authoring software and the specialised nature of its community have limited the availability of high-quality MEI encodings. Translating to MEI from other encoding formats, or generating MEI via optical music recognition processes, is thus a typical component of many MEI-project workflows. However, automated translations rarely achieve results of sufficient quality, a problem well-known in the community and documented in the literature. Final correction and validation by hand is therefore a common requirement. In this paper, we present mei-friend, an extension to the Atom text editor, which aims to relieve the degree of manual labour required in this process. The tool facilitates most common MEI editing tasks including the insertion and manipulation of MEI elements, makes the encoded score visible and interactively accessible to the user, and provides quality-of-life conveniences including keyboard shortcuts for editing functions as well as intelligent navigation of the MEI hierarchy. We detail the tool’s implementation, describe its functionalities, and evaluate its responsiveness during the editing process, even when editing very large MEI files.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Deep learning based source separation applied to choir ensembles

  • Petermann, Darius
  • Chandna, Pritish
  • Cuesta, Helena
  • Bonada, Jordi, 1973-
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Choral singing is a widely practiced form of ensemble singing wherein a group of people sing simultaneously in polyphonic harmony. The most commonly practiced setting for choir ensembles consists of four parts; Soprano, Alto, Tenor and Bass (SATB), each with its own range of fundamental frequencies (F0s). The task of source separation for this choral setting entails separating the SATB mixture into the constituent parts. Source separation for musical mixtures is well studied and many deep learning based methodologies have been proposed for the same. However, most of the research has been focused on a typical case which consists in separating vocal, percussion and bass sources from a mixture, each of which has a distinct spectral structure. In contrast, the simultaneous and harmonic nature of ensemble singing leads to high structural similarity and overlap between the spectral components of the sources in a choral mixture, making source separation for choirs a harder task than the typical case. This, along with the lack of an appropriate consolidated dataset has led to a dearth of research in the field so far. In this paper we first assess how well some of the recently developed methodologies for musical source separation perform for the case of SATB choirs. We then propose a novel domain-specific adaptation for conditioning the recently proposed U-Net architecture for musical source separation using the fundamental frequency contour of each of the singing groups and demonstrate that our proposed approach surpasses results from domain-agnostic architectures., The TITANX used for this research was donated by the NVIDIA Corporation. This work is partially supported by the Towards Richer Online Music Public-domain Archives (TROMPA H2020 770376) project. Helena Cuesta is supported by the FI Predoctoral Grant from AGAUR (Generalitat de Catalunya). The authors would like to thank Rodrigo Schramm and Emmanouil Benetos for sharing their singing voice datasets for this research.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Music emotion recognition: toward new, robust standards in personalized and context-sensitive applications

  • Gómez Cañón, Juan Sebastián
  • Cano, Estefanía
  • Eerola, Tuomas
  • Herrera Boyer, Perfecto, 1964-
  • Hu, Xiao
  • Yang, Yi-Hsuan
  • Gómez Gutiérrez, Emilia, 1975-
Emotion is one of the main reasons why people engage and interact with music [1] . Songs can express our inner feelings, produce goosebumps, bring us to tears, share an emotional state with a composer or performer, or trigger specific memories. Interest in a deeper understanding of the relationship between music and emotion has motivated researchers from various areas of knowledge for decades [2] , including computational researchers. Imagine an algorithm capable of predicting the emotions that a listener perceives in a musical piece, or one that dynamically generates music that adapts to the mood of a conversation in a film—a particularly fascinating and provocative idea. These algorithms typify music emotion recognition (MER), a computational task that attempts to automatically recognize either the emotional content in music or the emotions induced by music to the listener [3] . To do so, emotionally relevant features are extracted from music. The features are processed, evaluated, and then associated with certain emotions. MER is one of the most challenging high-level music description problems in music information retrieval (MIR), an interdisciplinary research field that focuses on the development of computational systems to help humans better understand music collections. MIR integrates concepts and methodologies from several disciplines, including music theory, music psychology, neuroscience, signal processing, and machine learning., Juan Gómez-Cañón is partially supported by the European Commission under the TROMPA project (H2020 770376).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Emotion annotation of music: a citizen science approach

  • Gutiérrez Páez, Nicolás Felipe
  • Gómez Cañón, Juan Sebastián
  • Porcaro, Lorenzo
  • Santos Rodríguez, Patrícia
  • Hernández Leo, Davinia
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: 27th International Conference, CollabTech 2021 celebrat de manera virtual del 31 d'agost al 3 de setembre de 2021., The understanding of the emotions in music has motivated research across diverse areas of knowledge for decades. In the field of computer science, there is a particular interest in developing algorithms to “predict” the emotions in music perceived by or induced to a listener. However, the gathering of reliable “ground truth” data for modeling the emotional content of music poses challenges, since tasks related with annotations of emotions are time consuming, expensive and cognitively demanding due to its inherent subjectivity and its cross-disciplinary nature. Citizen science projects have proven to be a useful approach to solve these types of problems where there is a need for recruiting collaborators for massive scale tasks. We developed a platform for annotating emotional content in musical pieces following a citizen science approach, to benefit not only the researchers, who benefit from the generated dataset, but also the volunteers, who are engaged to collaborate on the research project, not only by providing annotations but also through their self and community-awareness about the emotional perception of the music. Likewise, gamification mechanisms motivate the participants to explore and discover new music based on the emotional content. Preliminary user evaluations showed that the platform design is in line with the motivations of the general public, and that the citizen science approach offers an iterative refinement to enhance the quantity and quality of contributions by involving volunteers in the design process. The usability of the platform was acceptable, although some of the features require improvements., This work has been partially funded by the TROMPA project, European Union’s Horizon 2020 research and innovation programme under grant agreement no. 770376. TIDE-UPF also acknowledges the support by FEDER, the National Research Agency of the Spanish Ministry of Science and Innovation, TIN2017-85179-C3-3-R, PID2020-112584RB-C33, MDM-2015-0502, the Ramon y Cajal programme (P. Santos) and by ICREA under the ICREA Academia programme (D. Hernández-Leo, Serra Hunter).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Interweaving and enriching digital music collections for scholarship, performance, and enjoyment

  • Weigl, David M.
  • Goebl, Werner
  • Crawford, Tim
  • Gkiokas, Aggelos
  • Gutiérrez Páez, Nicolás Felipe
  • Porter, Alastair
  • Santos Rodríguez, Patrícia
  • Karreman, Casper
  • Vroomen, Ingmar
  • Liem, Cynthia C. S.
  • Sarasúa Berodia, Álvaro
  • van Tilburg, Marcel
Comunicació presentada a: 6th International Conference on Digital Libraries for Musicology (DLfM) celebrat el 9 de novembre de 2019 a La Haia, Països Baixos., The turn toward the digital has opened up previously difficult to access musical materials to wider musicological scholarship. Digital repositories provide access to publicly licensed score images, score encodings, textual resources, audiovisual recordings, and music metadata. While each repository reveals rich information for scholarly investigation, the unified exploration and analysis of separate digital collections remains a challenge. TROMPA—Towards Richer Online Music Public-domain Archives—addresses this through a knowledge graph interweaving composers, performers, and works described in established digital music libraries, facilitating discovery and combined access of complementary materials across collections. TROMPA provides for contribution of expert insights as citable, provenanced annotations, supporting analytical workflows and scholarly communication. Beyond scholars, the project targets four further user types: instrumental players; choir singers; orchestras; and music enthusiasts; with corresponding web applications providing specialised views of the same underlying knowledge graph. Thus, scholars’ annotations provide contextual information to other types of users; while performers’ rehearsal recordings and performative annotations, conductors’ marked up scores, and enthusiasts’ social discussions and listening behaviours, become available to scholarly analysis (per user consent). The knowledge graph is exposed as Linked Data, adhering to the FAIR principles of making data Findable, Accessible, Interoperable, and Re-usable, and supporting further interlinking, re-interpretation and re-use beyond the immediate scope of the project., This project has received funding from the European Union’s Horizon 2020 research and innovation programme H2020-EU.3.6.3.1. - Study European heritage, memory, identity, integration and cultural interaction and translation, including its representations in cultural and scientific collections, archives and museums, to better inform and understand the present by richer interpretations of the past under grant agreement No. 770376.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Content based singing voice extraction from a musical mixture

  • Chandna, Pritish
  • Blaauw, Merlijn
  • Bonada, Jordi, 1973-
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: ICASSP 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, celebrat en línia del 4 al 8 de maig de 2020., We present a deep learning based methodology for extracting the singing voice signal from a musical mixture based on the underlying linguistic content. Our model follows an encoder-decoder architecture and takes as input the magnitude component of the spectrogram of a musical mixture with vocals. The encoder part of the model is trained via knowledge distillation using a teacher network to learn a content embedding, which is decoded to generate the corresponding vocoder features. Using this methodology, we are able to extract the unprocessed raw vocal signal from the mixture even for a processed mixture dataset with singers not seen during training. While the nature of our system makes it incongruous with traditional objective evaluation metrics, we use subjective evaluation via listening tests to compare the methodology to state-of-the-art deep learning based source separation algorithms. We also provide sound examples and source code for reproducibility., This work is partially supported by the Towards Richer Online Music Public-domain Archives (TROMPA H2020 770376) project.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Personalized musically induced emotions of not-so-popular Colombian music

  • Gómez Cañón, Juan Sebastián
  • Herrera Boyer, Perfecto, 1964-
  • Cano, Estefanía
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada al workshop Human Centered AI inclòs a: 35th Conference on Neural Information Processing Systems (NeurIPS 2021) celebrat el 13 de desembre de manera virtual., This work presents an initial proof of concept of how Music Emotion Recognition (MER) systems could be intentionally biased with respect to annotations of musically-induced emotions in a political context. In specific, we analyze traditional Colombian music containing politically-charged lyrics of two types: (1) vallenatos and social songs from the “left-wing” guerrilla Fuerzas Armadas Revolucionarias de Colombia (FARC) and (2) corridos from the “right-wing” paramilitaries Autodefensas Unidas de Colombia (AUC). We train personalized machine learning models to predict induced emotions for three users with diverse political views – we aim at identifying the songs that may induce negative emotions for a particular user, such as anger and fear. To this extent, a user’s emotion judgements could be interpreted as problematizing data – subjective emotional judgments could in turn be used to influence the user in a human-centered machine learning environment. In short, highly desired “emotion regulation” applications could potentially deviate to “emotion manipulation” – the recent discredit of emotion recognition technologies might transcend ethical issues of diversity and inclusion., The research work conducted at the Universitat Pompeu Fabra is partially supported by the Eu- ropean Commission under the TROMPA project (H2020 770376) and the Project Musical AI - PID2019-111403GB-I00/AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Diversity in the music listening experience: insights from focus group interviews

  • Porcaro, Lorenzo
  • Gómez Gutiérrez, Emilia, 1975-
  • Castillo, Carlos
Comunicació presentada a: CHIIR '22: ACM SIGIR Conference on Human Information Interaction and Retrieval, celebrat del 14 al 18 de març de 2022 a Regensburg, Alemanya., Music listening in today’s digital spaces is highly characterized by the availability of huge music catalogues, accessible by people all over the world. In this scenario, recommender systems are designed to guide listeners in finding tracks and artists that best fit their requests, having therefore the power to influence the diversity of the music they listen to. Albeit several works have proposed new techniques for developing diversity-aware recommendations, little is known about how people perceive diversity while interacting with music recommendations. In this study, we interview several listeners about the role that diversity plays in their listening experience, trying to get a better understanding of how they interact with music recommendations. We recruit the listeners among the participants of a previous quantitative study, where they were confronted with the notion of diversity when asked to identify, from a series of electronic music lists, the most diverse ones according to their beliefs. As a follow-up, in this qualitative study we carry out semi-structured interviews to understand how listeners may assess the diversity of a music list and to investigate their experiences with music recommendation diversity. We report here our main findings on 1) what can influence the diversity assessment of tracks and artists’ music lists, and 2) which factors can characterize listeners’ interaction with music recommendation diversity., This work is partially supported by the European Commission under the TROMPA project (H2020 - grant agreement No. 770376). It is also part of the project Musical AI - PID2019-111403GB-I00/AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI). This work is also partially supported by the HUMAINT programme (Human Behaviour and Machine Intelligence), Joint Research Centre, European Commission. The project leading to these results received funding from "la Caixa" Foundation (ID 100010434), under the agreement LCF/PR/PR16/51110009.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A Vocoder based method for singing voice extraction

  • Chandna, Pritish
  • Blaauw, Merlijn
  • Bonada, Jordi, 1973-
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), celebrat del 12 al 17 de maig de 2019 a Brighton, Regne Unit., This paper presents a novel method for extracting the vocal track from a musical mixture. The musical mixture consists of a singing voice and a backing track which may comprise of various instruments. We use a convolutional network with skip and residual connections as well as dilated convolutions to estimate vocoder parameters, given the spectrogram of an input mixture. The estimated parameters are then used to synthesize the vocal track, without any interference from the backing track. We evaluate our system, through objective metrics pertinent to audio quality and interference from background sources, and via a comparative subjective evaluation. We use open-source source separation systems based on Non-negative Matrix Factorization (NMFs) and Deep Learning methods as benchmarks for our system and discuss future applications for this particular algorithm., The TITANX used for this research was donated by the NVIDIA Corporation. This work is partially supported by the Towards Richer Online Music Public-domain Archives (TROMPA) project.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

FAIR interconnection and enrichment of public-domain music resources on the Web

  • Weigl, David M.
  • Crawford, Tim
  • Gkiokas, Aggelos
  • Goebl, Werner
  • Gómez Gutiérrez, Emilia, 1975-
  • Gutiérrez Páez, Nicolás Felipe
  • Liem, Cynthia C. S.
  • Santos Rodríguez, Patrícia
Vast amounts of publicly licensed classical music resources are housed within many different repositories on the Web encompassing richly diverse facets of information—including bibliographical and biographical data, digitized images of music notation, music score encodings, audiovisual performance recordings, derived feature data, scholarly commentaries, and listener reactions. While these varied perspectives ought to contribute to greater holistic understanding of the music objects under consideration, in practice, such repositories are typically minimally connected. The TROMPA project aims to improve this situation by interconnecting and enriching public-domain music repositories. This is achieved, on the one hand, by the application of automated, cutting-edge Music Information Retrieval techniques, and on the other, by the development of contribution mechanisms enabling users to integrate their expertise. Information within established repositories is interrelated with data generated by the project within a data infrastructure whose design is guided by the FAIR principles of data management and stewardship: making music information Findable, Accessible, Interoperable, and Reusable. We provide an overview of challenges of description, identification, representation, contribution, and reliability toward applying the FAIR principles to music information, and outline TROMPA’s implementational approach to overcoming these challenges. This approach applies a graph-based data infrastructure to interrelate information hosted in different repositories on the Web within a unifying data model (a ‘knowledge graph’). Connections are generated across different representations of music content beyond the catalogue level, for instance connecting note elements within score encodings to corresponding moments in performance time-lines. Contributions of user data are supported via privacy-first mechanisms that retain control of such data with the contributing user. Provenance information is captured throughout, supporting reproducibility and re-use of the data both within and outside the context of the project., The TROMPA Project is funded under the European Union’s Horizon 2020 research and innovation programme H2020-EU.3.6.3.1. - Study European heritage, memory, identity, integration and cultural interaction and translation, including its representations in cultural and scientific collections, archives and museums, to better inform and understand the present by richer interpretations of the past under grant agreement No 770376.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

The AcousticBrainz genre dataset: Multi-source, multi-level, multi-label, and large-scale

  • Bogdanov, Dmitry
  • Porter, Alastair
  • Schreiber, Hendrik
  • Urbano, Julián
  • Oramas, Sergio
Comunicació presentada a: 20th International Society for Music Information Retrieval Conference celebrat del 4 al 8 de novembre de 2019 a Delft, Països Baixos., This paper introduces the AcousticBrainz Genre Dataset, a large-scale collection of hierarchical multi-label genre annotations from different metadata sources. It allows researchers to explore how the same music pieces are annotated differently by different communities following their own genre taxonomies, and how this could be addressed by genre recognition systems. Genre labels for the dataset are sourced from both expert annotations and crowds, permitting comparisons between strict hierarchies and folksonomies. Music features are available via the Acoustic- Brainz database. To guide research, we suggest a concrete research task and provide a baseline as well as an evaluation method. This task may serve as an example of the development and validation of automatic annotation algorithms on complementary datasets with different taxonomies and coverage. With this dataset, we hope to contribute to developments in content-based music genre recognition as well as cross-disciplinary studies on genre metadata analysis., This research has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreements No 688382 (AudioCommons) and 770376- 2 (TROMPA), as well as the Ministry of Economy and Competitiveness of the Spanish Government (Reference: TIN2015-69935-P).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

TROMPA-MER: an open dataset for personalized music emotion recognition

  • Gómez Cañón, Juan Sebastián
  • Gutiérrez Páez, Nicolás Felipe
  • Porcaro, Lorenzo
  • Porter, Alastair
  • Cano, Estefanía
  • Herrera Boyer, Perfecto, 1964-
  • Gkiokas, Aggelos
  • Santos Rodríguez, Patrícia
  • Hernández Leo, Davinia
  • Karreman, Casper
  • Gómez Gutiérrez, Emilia, 1975-
We present a platform and a dataset to help research on Music Emotion Recognition (MER). We developed the Music Enthusiasts platform aiming to improve the gathering and analysis of the so-called “ground truth” needed as input to MER systems. Firstly, our platform involves engaging participants using citizen science strategies and generate music emotion annotations – the platform presents didactic information and musical recommendations as incentivization, and collects data regarding demographics, mood, and language from each participant. Participants annotated each music excerpt with single free-text emotion words (in native language), distinct forced-choice emotion categories, preference, and familiarity. Additionally, participants stated the reasons for each annotation – including those distinctive of emotion perception and emotion induction. Secondly, our dataset was created for personalized MER and contains information from 181 participants, 4721 annotations, and 1161 music excerpts. To showcase the use of the dataset, we present a methodology for personalization of MER models based on active learning. The experiments show evidence that using the judgment of the crowd as prior knowledge for active learning allows for more effective personalization of MER systems for this particular dataset. Our dataset is publicly available and we invite researchers to use it for testing MER systems., This work was funded by the European Commission under the TROMPA project (H2020 770376) and the Project Musical AI - PID2019-111403GB-I00/AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Exploring artist gender bias in music recommendation

  • Shakespeare, Dougal
  • Porcaro, Lorenzo
  • Gómez Gutiérrez, Emilia, 1975-
  • Castillo, Carlos
Comunicació presentada a: Workshop on the Impact of Recommender Systems, ACM RecSys 2020 celebrat de manera virtual el 25 de setembre de 2020., Music Recommender Systems (mRS) are designed to give personalised and meaningful recommendations of items (i.e. songs, playlists or artists) to a user base, thereby reflecting and further complementing individual users’ specific music preferences. Whilst accuracy metrics have been widely applied to evaluate recommendations in mRS literature, evaluating a user’s item utility from other impactoriented perspectives, including their potential for discrimination, is still a novel evaluation practice in the music domain. In this work, we center our attention on a specific phenomenon for which we want to estimate if mRS may exacerbate its impact: gender bias. Our work presents an exploratory study, analyzing the extent to which commonly deployed state of the art Collaborative Filtering (CF) algorithms may act to further increase or decrease artist gender bias. To assess group biases introduced by CF, we deploy a recently proposed metric of bias disparity on two listening event datasets: the LFM-1b dataset, and the earlier constructed Celma’s dataset. Our work traces the causes of disparity to variations in input gender distributions and user-item preferences, highlighting the effect such configurations can have on user’s gender bias after recommendation generation., This work is partially supported by the European Commission under the TROMPA project (H2020 770376).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Personalisation and profiling using algorithms and not-so-popular Colombian music: goal-directed mechanisms in music emotion recognition

  • Gómez-Cañon, Juan Sebastián
  • Lennie, Thomas Magnus
  • Eerola, Tuomas
  • Aragón, Pablo
  • Cano, Estefanía
  • Herrera, Perfecto
  • Gómez, Emilia
This work investigates how personalised Music Emotion Recognition (MER) systems may lead to sensitive profiling when applied to musically induced emotions in politically charged contexts. We focus on traditional Colombian music with explicit political content, including (1) vallenatos and social songs aligned with the left-wing guerrilla Fuerzas Armadas Revolucionarias de Colombia (FARC), and (2) corridos linked to sympathisers of the right-wing paramilitary group Autodefensas Unidas de Colombia (AUC). Using data from 49 participants with diverse political leanings, we train personalised machine learning models to predict induced emotional responses -particularly negative emotions. Our findings reveal that political identity plays a significant role in shaping emotional experiences of music with explicit political content, and that emotion recognition models can capture this variation to a certain extent. These results raise critical concerns about the potential misuse of emotion recognition technologies. What is often framed as a tool for wellbeing and emotional regulation could, in politically sensitive contexts, be repurposed for user profiling. This work highlights the ethical risks of deploying AI-driven emotion analysis without safeguards, particularly among populations that are politically or socially vulnerable. We argue that subjective emotional responses may constitute sensitive personal data, and that failing to account for their sociopolitical context could amplify harm and exclusion., JSGC's contribution was partially supported by the European Commission through the TROMPA project (H2020 770376) and the project Musical AI (PID2019-111403GB-I00/AEI/10.13039/501100011033), funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI). TML was supported by the Arts and Humanities Research Council (AHRC) through a Northern Bridge Doctoral Training Partnership scholarship (grant code: AH/L503927/1).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Perceptions of diversity in electronic music: the impact of listener, artist, and track characteristics

  • Porcaro, Lorenzo
  • Gómez Gutiérrez, Emilia, 1975-
  • Castillo, Carlos
Shared practices to assess the diversity of retrieval system results are still debated in the Information Retrieval community, partly because of the challenges of determining what diversity means in specific scenarios, and of understanding how diversity is perceived by end-users. The field of Music Information Retrieval is not exempt from this issue. Even if fields such as Musicology or Sociology of Music have a long tradition in questioning the representation and the impact of diversity in cultural environments, such knowledge has not been yet embedded into the design and development of music technologies. In this paper, focusing on electronic music, we investigate the characteristics of listeners, artists, and tracks that are influential in the perception of diversity. Specifically, we center our attention on 1) understanding the relationship between perceived diversity and computational methods to measure diversity, and 2) analyzing how listeners' domain knowledge and familiarity influence such perceived diversity. To accomplish this, we design a user-study in which listeners are asked to compare pairs of lists of tracks and artists, and to select the most diverse list from each pair. We compare participants' ratings with results obtained through computational models built using audio tracks' features and artist attributes. We find that such models are generally aligned with participants' choices when most of them agree that one list is more diverse than the other, while they present a mixed behaviour in cases where participants have little agreement. Moreover, we observe how differences in domain knowledge, familiarity, and demographics can influence the level of agreement among listeners, and between listeners and diversity metrics computed automatically., This work is partially supported by the European Commission under the TROMPA project (H2020 - grant agreement No. 770376). This work is also partially supported by the HUMAINT programme (Human Behaviour and Machine Intelligence), Joint Research Centre, European Commission. The project leading to these results received funding from "la Caixa" Foundation (ID 100010434), under the agreement LCF/PR/PR16/51110009.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

WGansing: a multi-voice singing voice synthesizer based on the Wasserstein-Gan

  • Chandna, Pritish
  • Blaauw, Merlijn
  • Bonada, Jordi, 1973-
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada al EUSIPCO 2019: 27th European Signal Processing Conference, celebrat del 2 al 6 de setembre de 2019 a La Corunya, Espanya., We present a deep neural network based singing voice synthesizer, inspired by the Deep Convolutions Generative Adversarial Networks (DCGAN) architecture and optimized using the Wasserstein-GAN algorithm. We use vocoder parameters for acoustic modelling, to separate the influence of pitch and timbre. This facilitates the modelling of the large variability of pitch in the singing voice. Our network takes a block of consecutive frame-wise linguistic and fundamental frequency features, along with global singer identity as input and outputs vocoder features, corresponding to the block of features. This block-wise approach, along with the training methodology allows us to model temporal dependencies within the features of the input block. For inference, sequential blocks are concatenated using an overlap-add procedure. We show that the performance of our model is competitive with regards to the state-of-the-art and the original sample using objective metrics and a subjective listening test. We also present examples of the synthesis on a supplementary website and the source code via GitHub., This work is partially supported by the European Commission under the TROMPA project (H2020 770376). The TITAN X used for this research was donated by the NVIDIA Corporation.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Accurate and scalable version identification using musically-motivated embeddings

  • Yesiler, Furkan
  • Serrà Julià, Joan
  • Gómez Gutiérrez, Emilia, 1975-
The version identification (VI) task deals with the automatic detection of recordings that correspond to the same underlying musical piece. Despite many efforts, VI is still an open problem, with much room for improvement, specially with regard to combining accuracy and scalability. In this paper, we present MOVE, a musically-motivated method for accurate and scalable version identification. MOVE achieves state-of-the-art performance on two publicly-available benchmark sets by learning scalable embeddings in an Euclidean distance space, using a triplet loss and a hard triplet mining strategy. It improves over previous work by employing an alternative input representation, and introducing a novel technique for temporal content summarization, a standardized latent space, and a data augmentation strategy specifically designed for VI. In addition to the main results, we perform an ablation study to highlight the importance of our design choices, and study the relation between embedding dimensionality and model performance., Comunicació presentada a: ICASSP 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, celebrat en línia del 4 al 8 de maig de 2020., This work is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skodowska-Curie grant agreement No. 765068, and by TROMPA, the Horizon 2020 project 770376-2.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

The MediaEval 2018 AcousticBrainz genre task: content-based music genre recognition from multiple sources

  • Bogdanov, Dmitry
  • Porter, Alastair
  • Urbano, Julián
  • Schreiber, Hendrik
Comunicació presentada al MediaEval 2018 Workshop celebrat a Sophia Antipolis (França) del 29 al 31 d'octubre de 2018., This paper provides an overview of the AcousticBrainz Genre Task organized as part of the MediaEval 2018 Benchmarking Initiative for Multimedia Evaluation. The task is focused on content-based music genre recognition using genre annotations from multiple sources and large-scale music features data available in the AcousticBrainz database. The goal of our task is to explore how the same music pieces can be annotated differently by different communities following different genre taxonomies, and how this should be addressed by content-based genre recognition systems. We present the task challenges, the employed ground-truth information and datasets, and the evaluation methodology., We thank all contributors to AcousticBrainz. This research has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreements No 688382 (AudioCommons) and 770376-2 (TROMPA), as well as the Ministry of Economy and Competitiveness of the Spanish Government (Reference: TIN2015-69935-P). We also thank tagtraum industries for providing the Tagtraum genre annotations.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Music recommendation diversity: a tentative framework and preliminary results

  • Porcaro, Lorenzo
  • Castillo, Carlos
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada al 1st Workshop on Designing Human-Centric MIR Systems celebrat el 2 de novembre de 2019 a Delft, Països Baixos, en el marc del 20th annual conference of the International Society for Music Information Retrieval (ISMIR)., Music recommendations are increasingly part of the listening experience of people all over the world, especially in the context of streaming services. In this scenario, recommender systems’ role is to help users in finding music that can fit their interests and tastes. However, Westerncentric perspectives in systems’ design are often subject to criticisms because of their power of reinforcing already existing cultural bias and therefore potentially impacting negatively on the music distribution mechanisms. In our research proposal, we aim to address the problem of assessing the impact of music recommendation diversity, or the lack thereof. This requires 1) the formalization of a working definition of diversity in the music field 2) the development of evaluation practices for estimating diversity in the context of music recommender systems 3) the observation of emerging impact due to music recommendation diversity 4) the proposal of countermeasures for mitigating negative or reinforcing positive impact observed. Basing on already known consequences of information technologies in political, economic and social areas, our goal is to understand the cultural impact that music recommender systems can have on our society., This work is partially supported by the European Commission under the TROMPA project (H2020 770376).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Cough-based COVID-19 detection with contextual attention convolutional neural networks and gender information

  • Mallol Ragolta, Adrià
  • Cuesta, Helena
  • Gómez Gutiérrez, Emilia, 1975-
  • Schuller, Björn
Comunicació presentada a INTERSPEECH 2021, celebrat del 30 d'agost al 3 de setembre de 2021 a Brno, Txèquia., The aim of this contribution is to automatically detect COVID19 patients by analysing the acoustic information embedded in coughs. COVID-19 affects the respiratory system, and, consequently, respiratory-related signals have the potential to contain salient information for the task at hand. We focus on analysing the spectrogram representations of cough samples with the aim to investigate whether COVID-19 alters the frequency content of these signals. Furthermore, this work also assesses the impact of gender in the automatic detection of COVID-19. To extract deep-learnt representations of the spectrograms, we compare the performance of a cough-specific, and a Resnet18 pre-trained Convolutional Neural Network (CNN). Additionally, our approach explores the use of contextual attention, so the model can learn to highlight the most relevant deep-learnt features extracted by the CNN. We conduct our experiments on the dataset released for the Cough Sound Track of the DICOVA 2021 Challenge. The best performance on the test set is obtained using the Resnet18 pre-trained CNN with contextual attention, which scored an Area Under the Curve (AUC) of 70.91 % at 80 % sensitivity., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreements No. 826506 (sustAGE) and No. 770376 (TROMPA). Further funding has been received from the FI Predoctoral Grant 2018FI-B01015 from AGAUR, Generalitat de Catalunya.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Recognizing musical entities in user-generated content

  • Porcaro, Lorenzo
  • Saggion, Horacio
Comunicació presentada a: 20th International Conference on Computational Linguistics and Intelligent Text Processing (CICLing) celebrat del 7 al 13 d'abril de 2019 a La Rochelle, França., Recognizing Musical Entities is important for Music Information Retrieval (MIR) since it can improve the performance of several tasks such as music recommendation, genre classification or artist similarity. However, most entity recognition systems in the music domain have concentrated on formal texts (e.g. artists’ biographies, encyclopedic articles, etc.), ignoring rich and noisy user-generated content. In this work, we present a novel method to recognize musical entities in Twitter content generated by users following a classical music radio channel. Our approach takes advantage of both formal radio schedule and users’ tweets to improve entity recognition. We instantiate several machine learning algorithms to perform entity recognition combining task-specific and corpus-based features. We also show how to improve recognition results by jointly considering formal and user-generated content., This work is partially supported by the European Commission under the TROMPA project (H2020 770376), and by the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015- 0502).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

20 years of playlists: A statistical analysis on popularity and diversity

  • Porcaro, Lorenzo
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: 20th annual conference of the International Society for Music Information Retrieval (ISMIR) celebrart del 4 al 8 de novembre de 2019 a Delft, Països Baixos., Grouping songs together, according to music preferences, mood or other characteristics, is an activity which reflects personal listening behaviours and tastes. In the last two decades, due to the increasing size of music catalogue accessible and to improvements of recommendation algorithms, people have been exposed to new ways for creating playlists. In this work, through the statistical analysis of more than 400K playlists from four datasets, created in different temporal and technological contexts, we aim to understand if it is possible to extract information about the evolution of humans strategies for playlist creation. We focus our analysis on two driving concepts of the Music Information Retrieval literature: popularity and diversity., This work is partially supported by the European Commission under the TROMPA project (H2020 770376).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

How diverse is the ACII community? Analysing gender, geographical and business diversity of affective computing research

  • Hupont, Isabelle
  • Tolan, Songül
  • Freire, Ana
  • Porcaro, Lorenzo
  • Estevez, Sara
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a 2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII), celebrat del 28 de setembre a l'1 d'octubre de 2021 de manera virtual., ACII is the premier international forum for presenting the latest research on affective computing. In this work, we monitor, quantify and reflect on the diversity in ACII conference across time by computing a set of indexes. We measure diversity in terms of gender, geographic location and academia vs research centres vs industry, and consider three different actors: authors, keynote speakers and organizers. Results raise awareness on the limited diversity in the field, in all studied facets, and compared to other AI conferences. While gender diversity is relatively high, equality is far from being reached. The community is dominated by European, Asian and North American researchers, leading the rest of continents under-represented. There is also a strong absence of companies and research centres focusing on applied research and products. This study fosters discussion in the community on the need for diversity and related challenges in terms of minimizing potential biases of the developed systems to the represented groups. We intend our paper to contribute with a first analysis to consider as a monitoring tool when implementing diversity initiatives. The data collected for this study are publicly released through the European divinAI initiative., This work is partially supported by the European Commission under the TROMPA project (H2020 770376) and the HUMAINT project of the Joint Research Centre.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

End-to-end sound source separation conditioned on instrument labels

  • Slizovskaia, Olga
  • Kim, Leo
  • Haro Ortega, Gloria
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing celebrat de 12 al 17 de maig de 2019 a Brighton, Regne Unit., Can we perform an end-to-end music source separation with a variable number of sources using a deep learning model? This paper presents an extension of the Wave-UNet [1] model which allows end-to-end monaural source separation with a non-fixed number of sources. Furthermore, we propose multiplicative conditioning with instrument labels at the bottleneck of the Wave-U-Net and show its effect on the separation results. This approach can be further extended to other types of conditioning such as audio-visual source separation and score-informed source separation., This work has received funding from the Maria de Maeztu Programme (MDM-2015-0502), ERC Innovation Programme (grant 770376, TROMPA), and MINECO/FEDER UE project (TIN2015-70410-C2-1-R).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A deep learning based analysis-synthesis framework for unison singing

  • Chandna, Pritish
  • Cuesta, Helena
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Unison singing is the name given to an ensemble of singers simultaneously singing the same melody and lyrics. While each individual singer in a unison sings the same principle melody, there are slight timing and pitch deviations between the singers, which, along with the ensemble of timbres, give the listener a perceived sense of "unison". In this paper, we present a study of unison singing in the context of choirs; utilising some recently proposed deep-learning based methodologies, we analyse the fundamental frequency (F0) distribution of the individual singers in recordings of unison mixtures. Based on the analysis, we propose a system for synthesising a unison signal from an a cappella input and a single voice prototype representative of a unison mixture. We use subjective listening test to evaluate perceptual factors of our proposed system for synthesis, including quality, adherence to the melody as well the degree of perceived unison., The TITANX used for this research was donated by the NVIDIA Corporation. This work is partially supported by the Towards Richer Online Music Public-domain Archives (TROMPA H2020 770376) project. Helena Cuesta is supported by the FI Predoctoral Grant from AGAUR (Generalitat de Catalunya).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Da-TACOS: A dataset for cover song identification and understanding

  • Yesiler, Furkan
  • Tralie, Chris
  • Correya, Albin Andrew
  • Silva, Diego F.
  • Tovstogan, Philip
  • Gómez Gutiérrez, Emilia, 1975-
  • Serra, Xavier
Comunicació presentada a: 20th annual conference of the International Society for Music Information Retrieval (ISMIR) celebrat del 4 al 8 de novembre de 2019 a Delft, Països Baixos., This paper focuses on Cover Song Identification (CSI), an important research challenge in content-based Music Information Retrieval (MIR). Although the task itself is interesting and challenging for both academia and industry scenarios, there are a number of limitations for the advancement of current approaches. We specifically address two of them in the present study. First, the number of publicly available datasets for this task is limited, and there is no publicly available benchmark set that is widely used among researchers for comparative algorithm evaluation. Second, most of the algorithms are not publicly shared and reproducible, limiting the comparison of approaches. To overcome these limitations we propose Da-TACOS, a DaTAset for COver Song Identification and Understanding, and two frameworks for feature extraction and benchmarking to facilitate reproducibility. Da-TACOS contains 25K songs represented by unique editorial metadata plus 9 low- and mid-level features pre-computed with open source libraries, and is divided into two subsets. The Cover Analysis subset contains audio features (e.g. key, tempo) that can serve to study how musical characteristics vary for cover songs. The Benchmark subset contains the set of features that have been frequently used in CSI research, e.g. chroma, MFCC, beat onsets etc. Moreover, we provide initial benchmarking results of a selected number of state-of-the-art CSI algorithms using our dataset, and for reproducibility, we share a GitHub repository containing the feature extraction and benchmarking frameworks., This work is partially supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068, and by TROMPA, the Horizon 2020 project 770376-2.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Analysis of intonation in unison choir singing

  • Cuesta, Helena
  • Gómez Gutiérrez, Emilia, 1975-
  • Martorell Domínguez, Agustín
  • Loáiciga, Felipe
Comunicació presentada a la 15th International Conference on Music Perception and Cognition celebrada del 23 al 28 de juliol a Graz, Àustria., Choir singers synchronize their fundamental frequency (f0) and timing when they perform together. In this work, we investigate several expressive characteristics of choir singing, with special emphasis on unison performances, to study how singers blend together and interact with each other by analyzing their f0 dispersion, intonation, and vibrato. We use the spectral autocorrelation algorithm to obtain the f0 trajectories of each individual singer and compute the f0 dispersion as the standard deviation of the f0 distribution. Using the Pearson correlation, we study the synchronization of singers in terms of their intonation and vibrato. Together with this study, we present a new choir singing dataset of voices singing together but recorded individually using directional close microphones. This dataset contains the individual recordings of 16 singers from a semiprofessional 4-part choir (SATB) performing 3 different songs. We found the f0 dispersion in unison sections to be in the range of 16 to 30 cents, and the largest average f0 dispersion was found in the basses. We observed that the intonation correlation varies a lot and is around 0.32 on average in all cases. The slowest piece was the one with the highest percentage of notes with vibrato, which had a frequency around 5 Hz for the whole dataset., This work is partially supported by the Spanish Ministry of Economy and Competitiveness under the CASAS project (TIN2015-70816-R) and the European Commission under the TROMPA project (H2020 770376).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Multiple F0 estimation in vocal ensembles using convolutional neural networks

  • Cuesta, Helena
  • McFee, Brian
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., This paper addresses the extraction of multiple F0 values from polyphonic and a cappella vocal performances using convolutional neural networks (CNNs). We address the major challenges of ensemble singing, i.e., all melodic sources are vocals and singers sing in harmony. We build upon an existing architecture to produce a pitch salience function of the input signal, where the harmonic constantQ transform (HCQT) and its associated phase differentials are used as an input representation. The pitch salience function is subsequently thresholded to obtain a multiple F0 estimation output. For training, we build a dataset that comprises several multi-track datasets of vocal quartets with F0 annotations. This work proposes and evaluates a set of CNNs for this task in diverse scenarios and data configurations, including recordings with additional reverb. Our models outperform a state-of-the-art method intended for the same music genre when evaluated with an increased F0 resolution, as well as a general-purpose method for multi-F0 estimation. We conclude with a discussion on future research directions., The authors would like to thank Rodrigo Schramm and Emmanouil Benetos for sharing the BSQ and BC datasets for this research. Helena Cuesta is supported by the FI Predoctoral Grant from AGAUR (Generalitat de Catalunya). This work is partially supported by the European Commission under the TROMPA project (H2020 770376) and MARL-NYU (as part of a two-months research stay).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Conditioned source separation for musical instrument performances

  • Slizovskaia, Olga
  • Haro Ortega, Gloria
  • Gómez Gutiérrez, Emilia, 1975-
In music source separation, the number of sources may vary for each piece and some of the sources may belong to the same family of instruments, thus sharing timbral characteristics and making the sources more correlated. This leads to additional challenges in the source separation problem. This paper proposes a source separation method for multiple musical instruments sounding simultaneously and explores how much additional information apart from the audio stream can lift the quality of source separation. We explore conditioning techniques at different levels of a primary source separation network and utilize two extra modalities of data, namely presence or absence of instruments in the mixture, and the corresponding video stream data., This work was funded in part by ERC Innovation Programme (grant 770376, TROMPA); Spanish Ministry of Economy and Competitiveness under the Mar´ıa de Maeztu Units of Excellence Program (MDM-2015-0502) and the Social European Funds; the MICINN/FEDER UE project with reference PGC2018-098625-B-I00; and the H2020-MSCARISE-2017 project with reference 777826 NoMADS. We gratefully acknowledge NVIDIA for the donation of GPUs used for the experiments.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Sequence-to-sequence singing synthesis using the feed-forward transformer

  • Blaauw, Merlijn
  • Bonada, Jordi, 1973-
Comunicació presentada a: ICASSP 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, celebrat en línia del 4 al 8 de maig de 2020., We propose a sequence-to-sequence singing synthesizer, which avoids the need for training data with pre-aligned phonetic and acoustic features. Rather than the more common approach of a content-based attention mechanism combined with an autoregressive decoder, we use a different mechanism suitable for feed-forward synthesis. Given that phonetic timings in singing are highly constrained by the musical score, we derive an approximate initial alignment with the help of a simple duration model. Then, using a decoder based on a feed-forward variant of the Transformer model, a series of self-attention and convolutional layers refines the result of the initial alignment to reach the target acoustic features. Advantages of this approach include faster inference and avoiding the exposure bias issues that affect autoregressive models trained by teacher forcing. We evaluate the effectiveness of this model compared to an autoregressive baseline, the importance of self-attention, and the importance of the accuracy of the duration model., This work was funded by TROMPA H2020 No 770376.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A Model for evaluating popularity and semantic information variations in radio listening sessions

  • Porcaro, Lorenzo
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: 1st Workshop on the Impact of Recommender Systems dins el 13th ACM Conference on Recommender Systems (RecSys 2019) celebrat el 19 de setembre de 2019 a Copenhagen, Dinamarca., Listening to music radios is an activity that since the 20th century is part of the cultural habits for people all over the world. While in the case of analog radios DJs are in charge of selecting the music to be broadcasted, nowadays recommender systems analyzing users’ behaviours can automatically generate radios tailored to users’ musical taste. Nonetheless, in both cases listening sessions do not depend on the listener choices, but on a set of external recommendations received. In this preliminary study, we propose a model for estimating features’ variation during listening sessions, comparing different scenarios, namely analog radios, personalized and not-personalized streaming radios. In particular, we focus on the analysis of track popularity and semantic information, features well-established in the Music Information Retrieval literature. The presented model aims to quantify the possible impacts of the sessions’ variation on the user listening experience., This work is partially supported by the European Commission under the TROMPA project (H2020 770376).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A deep-learning based framework for source separation, analysis, and synthesis of choral ensembles

  • Chandna, Pritish
  • Cuesta, Helena
  • Petermann, Darius
  • Gómez Gutiérrez, Emilia, 1975-
Choral singing in the soprano, alto, tenor and bass (SATB) format is a widely practiced and studied art form with significant cultural importance. Despite the popularity of the choral setting, it has received little attention in the field of Music Information Retrieval. However, the recent publication of high-quality choral singing datasets as well as recent developments in deep learning based methodologies applied to the field of music and speech processing, have opened new avenues for research in this field. In this paper, we use some of the publicly available choral singing datasets to train and evaluate state-of-the-art source separation algorithms from the speech and music domains for the case of choral singing. Furthermore, we evaluate existing monophonic F0 estimators on the separated unison stems and propose an approximation of the perceived F0 of a unison signal. Additionally, we present a set of applications combining the proposed methodologies, including synthesizing a single singer voice from the unison, and transposing and remixing the separated stems into a synthetic multi-singer choral signal. We finally conduct a set of listening tests to perform a perceptual evaluation of the results we obtain with the proposed methodologies., This work is partially supported by the European Commission under the TROMPA project (H2020 770376), and the project Musical AI (PID 2019-111403GB-I00/AEI/10.13039/501100011033) funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Less is more: faster and better music version identification with embedding distillation

  • Serrà Julià, Joan
  • Yesiler, Furkan
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Version identification systems aim to detect different renditions of the same underlying musical composition (loosely called cover songs). By learning to encode entire recordings into plain vector embeddings, recent systems have made significant progress in bridging the gap between accuracy and scalability, which has been a key challenge for nearly two decades. In this work, we propose to further narrow this gap by employing a set of data distillation techniques that reduce the embedding dimensionality of a pre-trained state-of-the-art model. We compare a wide range of techniques and propose new ones, from classical dimensionality reduction to more sophisticated distillation schemes. With those, we obtain 99% smaller embeddings that, moreover, yield up to a 3% accuracy increase. Such small embeddings can have an important impact in retrieval time, up to the point of making a real-world system practical on a standalone laptop., This work is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068, and by TROMPA, the Horizon 2020 project 770376-2.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A Framework for multi-f0 modeling in SATB choir recordings

  • Cuesta, Helena
  • Gómez Gutiérrez, Emilia, 1975-
  • Chandna, Pritish
Comunicació presentada a: Sound & Music Computing Conference celebrada del 28 al 31 de maig de 2019 a Màlaga, Espanya., Fundamental frequency (f0) modeling is an important but relatively unexplored aspect of choir singing. Performance evaluation as well as auditory analysis of singing, whether individually or in a choir, often depend on extracting f0 contours for the singing voice. However, due to the large number of singers, singing at a similar frequency range, extracting the exact individual pitch contours from choir recordings is a challenging task. In this paper, we address this task and develop a methodology for modeling pitch contours of SATB choir recordings. A typical SATB choir consists of four parts, each covering a distinct range of pitches and often with multiple singers each. We first evaluate some state-of-the-art multi-f0 estimation systems for the particular case of choirs with a single singer per part, and observe that the pitch of individual singers can be estimated to a relatively high degree of accuracy. We observe, however, that the scenario of multiple singers for each choir part (i.e. unison singing) is far more challenging. In this work we propose a methodology based on combining a multi-f0 estimation methodology based on deep learning followed by a set of traditional DSP techniques to model f0 and its dispersion instead of a single f0 trajectory for each choir part. We present and discuss our observations and test our framework with different singer configurations., This work is partially supported by the European Commission under the TROMPA project (H2020 770376) and by Spanish Ministry of Economy and Competitiveness under the CASAS project (TIN2015-70816-R). First author is supported by FI Predoctoral Grant from AGAUR (Generalitat de Catalunya).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Combining musical features for cover detection

  • Doras, Guillaume
  • Yesiler, Furkan
  • Serrà Julià, Joan
  • Gómez Gutiérrez, Emilia, 1975-
  • Peeters, Geoffroy
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Recent works have addressed the automatic cover detection problem from a metric learning perspective. They employ different input representations, aiming to exploit melodic or harmonic characteristics of songs and yield promising performances. In this work, we propose a comparative study of these different representations and show that systems combining melodic and harmonic features drastically outperform those relying on a single input representation. We illustrate how these features complement each other with both quantitative and qualitative analyses. We finally investigate various fusion schemes and propose methods yielding state-of-the-art performances on two publicly-available large datasets., FY is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068, and EG by TROMPA, the Horizon 2020 project 770376-2.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

MediaEval 2018 AcousticBrainz genre task: A baseline combining deep feature embeddings across datasets

  • Oramas, Sergio
  • Bogdanov, Dmitry
  • Porter, Alastair
Comunicació presentada al MediaEval 2018 Workshop celebrat a Sophia Antipolis (França) del 29 al 31 d'octubre de 2018., In this paper we present a baseline approach for the MediaEval 2018 AcousticBrainz Genre Task that takes advantage of stacking multiple feature embeddings learned on individual genre datasets by simple deep learning architectures. Although we employ basic neural networks, the combination of their deep feature embeddings provides a significant gain in performance compared to each individual network., This research has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreements No 688382 (AudioCommons) and 770376-2 (TROMPA), as well as the Ministry of Economy and Competitiveness of the Spanish Government (Reference: TIN2015-69935-P).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Voice assignment in vocal quartets using deep learning models based on pitch salience

  • Cuesta, Helena
  • Gómez Gutiérrez, Emilia, 1975-
This paper deals with the automatic transcription of four-part, a cappella singing, audio performances. In particular, we exploit an existing, deep-learning based, multiple F0 estimation method and complement it with two neural network architectures for voice assignment (VA) in order to create a music transcription system that converts an input audio mixture into four pitch contours. To train our VA models, we create a novel synthetic dataset by collecting 5381 choral music scores from public-domain music archives, which we make publicly available for further research. We compare the performance of the proposed VA models on different types of input data, as well as to a hidden Markov model-based baseline system. In addition, we assess the generalization capabilities of these models on audio recordings with differing pitch distributions and vocal music styles. Our experiments show that the two proposed models, a CNN and a ConvLSTM, have very similar performance, and both of them outperform the baseline HMM-based system. We also observe a high confusion rate between the alto and tenor voice parts, which commonly have overlapping pitch ranges, while the bass voice has the highest scores in all evaluated scenarios., This work is partially supported by the European Commission under the TROMPA project (H2020 770376), the Spanish Ministry of Science and Innovation under the Musical AI project (PID2019-111403GB-I00), and by AGAUR (Generalitat de Catalunya) through an FI Predoctoral Grant (2018FI-B01015).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Transfer learning from speech to music: towards language-sensitive emotion recognition models

  • Gómez Cañón, Juan Sebastián
  • Cano, Estefanía
  • Herrera Boyer, Perfecto, 1964-
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: 28th European Signal Processing Conference (EUSIPCO 2020) celebrat del 18 al 22 de gener de 2021 a Amsterdam, Països Baixos., In this study, we address emotion recognition using unsupervised feature learning from speech data, and test its transferability to music. Our approach is to pre-train models using speech in English and Mandarin, and then fine-tune them with excerpts of music labeled with categories of emotion. Our initial hypothesis is that features automatically learned from speech should be transferable to music. Namely, we expect the intra-linguistic setting (e.g., pre-training on speech in English and fine-tuning on music in English) should result in improved performance over the cross-linguistic setting (e.g., pre-training on speech in English and fine-tuning on music in Mandarin). Our results confirm previous research on cross-domain transferability, and encourage research towards language-sensitive Music Emotion Recognition (MER) models., The research work conducted in the Music Technology Group at the Universitat Pompeu Fabra is partially supported by the European Commission under the TROMPA project (H2020 770376).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Diversity by design in music recommender systems

  • Porcaro, Lorenzo
  • Castillo, Carlos
  • Gómez Gutiérrez, Emilia, 1975-
Music Recommender Systems (Music RS) are nowadays pivotal in shaping the listening experience of people all around the world. Partly driven by the commercial application of this technology, music recommendation research has gained increasing attention both within and outside the Music Information Retrieval (MIR) community. Thanks also to the widespread use of recommender systems in music streaming services, it has been possible to enhance several characteristics of such systems in terms of performance, design, and user experience. Nonetheless, imagining Music RS only from an application-driven perspective may generate an incomplete view of how this technology is affecting people’s habitus, from the decision-making processes to the formation of musical taste and opinions. In this overview, we address the concept of diversity in music recommendation, and taking a value-driven approach we review diversity-related methodologies proposed in the Music RS literature. Additionally, by taking as an example the wider context of Information Technology (IT), we present the elements interacting in the diversity by design paradigm. We do that to acknowledge the lack of a comprehensive framework in Music RS research to address diversity, until now mostly driven by empirical results and fragmented in different application areas. Maintaining an interdisciplinary perspective, we discuss some challenges that MIR practitioners may face when researching Music RS, going beyond the search for better performance and instead questioning the theoretical foundations on which to base future research., This work is partially supported by the European Commission under the TROMPA project (H2020 – grant agreement No. 770376). This work is also partially supported by the HUMAINT programme (Human Behaviour and Machine Intelligence), Joint Research Centre, European Commission. The project leading to these results received funding from “la Caixa” Foundation (ID 100010434), under the agreement LCF/PR/PR16/51110009.