Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 36
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Data augmentation for instrument classification robust to audio effects

  • Ramires, António
  • Serra, Xavier
Comunicació presentada a la 22a International Conference on Digital Audio Effects (DAFx-19) que se celebra del 2 al 6 de setembre de 2019 a Birmingham, Regne Unit., Reusing recorded sounds (sampling) is a key component in Electronic Music Production (EMP), which has been present since its early days and is at the core of genres like hip-hop or jungle. Commercial and non-commercial services allow users to obtain collections of sounds (sample packs) to reuse in their compositions. Automatic classification of one-shot instrumental sounds allows automatically categorising the sounds contained in these collections, allowing easier navigation and better characterisation. Automatic instrument classification has mostly targeted the classification of unprocessed isolated instrumental sounds or detecting predominant instruments in mixed music tracks. For this classification to be useful in audio databases for EMP, it has to be robust to the audio effects applied to unprocessed sounds. In this paper we evaluate how a state of the art model trained with a large dataset of one-shot instrumental sounds performs when classifying instruments processed with audio effects. In order to evaluate the robustness of the model, we use data augmentation with audio effects and evaluate how each effect influences the classification accuracy., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 765068, MIP-Frontiers. We thank Matthew Davies for reviewing a draft of this paper and providing helpful feedback.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

The Freesound Loop dataset and annotation tool

  • Ramires, António
  • Font Corbera, Frederic
  • Bogdanov, Dmitry
  • Smith, Jordan B. L.
  • Yang, Yi-Hsuan
  • Ching, Joann
  • Chen, Bo-Yu
  • Wu, Yueh-Kao
  • Wei-Han, Hsu
  • Serra, Xavier
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Music loops are essential ingredients in electronic music production, and there is a high demand for pre-recorded loops in a variety of styles. Several commercial and community databases have been created to meet this demand, but most are not suitable for research due to their strict licensing. We present the Freesound Loop Dataset (FSLD), a new large-scale dataset of music loops annotated by experts. The loops originate from Freesound, a community database of audio recordings released under Creative Commons licenses, so the audio in our dataset may be redistributed. The annotations include instrument, tempo, meter, key and genre tags. We describe the methodology used to assemble and annotate the data, and report on the distribution of tags in the data and inter-annotator agreement. We also present to the community an online loop annotator tool that we developed. To illustrate the usefulness of FSLD, we present short case studies on using it to estimate tempo and key, generate music tracks, and evaluate a loop separation algorithm. We anticipate that the community will find yet more uses for the data, in applications from automatic loop characterisation to algorithmic composition., This research was funded in part by European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No765068, MIP-Frontiers and by a grant from the Ministry of Science and Technology, Taiwan (MOST107-2221-E-001-013-MY2).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Assessing algorithmic biases for musical version identification

  • Yesiler, Furkan
  • Serrà, Joan
  • Miron, Marius
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a 15th ACM International Conference on Web Search and Data Mining (WSDM 2022), celebrat del 21 al 25 de febrer de 2022 de manera virtual., Version identification (VI) systems now offer accurate and scalable solutions for detecting different renditions of a musical composition, allowing the use of these systems in industrial applications and throughout the wider music ecosystem. Such use can have an important impact on various stakeholders regarding recognition and financial benefits, including how royalties are circulated for digital rights management. In this work, we take a step toward acknowledging this impact and consider VI systems as socio-technical systems rather than isolated technologies. We propose a framework for quantifying performance disparities across 5 systems and 6 relevant side attributes: gender, popularity, country, language, year, and prevalence. We also consider 3 main stakeholders for this particular information retrieval use case: the performing artists of query tracks, those of reference (original) tracks, and the composers. By categorizing the recordings in our dataset using such attributes and stakeholders, we analyze whether the considered VI systems show any implicit biases. We find signs of disparities in identification performance for most of the groups we include in our analyses. We also find that learning- and rule-based systems behave differently for some attributes, which suggests an additional dimension to consider along with accuracy and scalability when evaluating VI systems. Lastly, we share our dataset to encourage VI researchers to take these aspects into account while building new systems., This work is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Accurate and scalable version identification using musically-motivated embeddings

  • Yesiler, Furkan
  • Serrà Julià, Joan
  • Gómez Gutiérrez, Emilia, 1975-
The version identification (VI) task deals with the automatic detection of recordings that correspond to the same underlying musical piece. Despite many efforts, VI is still an open problem, with much room for improvement, specially with regard to combining accuracy and scalability. In this paper, we present MOVE, a musically-motivated method for accurate and scalable version identification. MOVE achieves state-of-the-art performance on two publicly-available benchmark sets by learning scalable embeddings in an Euclidean distance space, using a triplet loss and a hard triplet mining strategy. It improves over previous work by employing an alternative input representation, and introducing a novel technique for temporal content summarization, a standardized latent space, and a data augmentation strategy specifically designed for VI. In addition to the main results, we perform an ablation study to highlight the importance of our design choices, and study the relation between embedding dimensionality and model performance., Comunicació presentada a: ICASSP 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, celebrat en línia del 4 al 8 de maig de 2020., This work is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skodowska-Curie grant agreement No. 765068, and by TROMPA, the Horizon 2020 project 770376-2.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Similarity of nearest-neighbor query results in deep latent spaces

  • Tovstogan, Philip
  • Serra, Xavier
  • Bogdanov, Dmitry
Comunicació presentada a: 19th Sound and Music Computing Conference, celebrat del 5 al 12 de juny de 2022 a Sant-Étienne, Music recommendation systems are commonly used for personalized recommendations. However, there are cases where due to privacy concerns or design decisions, there is no user information nor collaborative filtering data available. In those cases, it is possible to use content-based similarity spaces to retrieve the most similar tracks to be recommended based on the reference track. In this paper, we compare the latent spaces extracted from state-of-the-art autotagging models in terms of the similarity between lists of retrieved nearest neighbors. We additionally study item factors from collaborative-filtering data as a reference. We provide insights into how much the choice of the architecture, training dataset, or model layer (output vs. penultimate) as well as a projection of the latent space onto 2D changes the list of retrieved nearest neighbors. We release the dataset of 9 content-based and 3 collaborative-filtering latent representations of 29 275 tracks from Jamendo that we use for the evaluation. Moreover, we perform an online user experiment to compare the perceived track-to-track similarity of the selected evaluated latent spaces. The results show that content-based spaces show better results in our scenario, particularly embeddings from penultimate layers of auto-tagging architectures., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skøodowska-Curie grant agreement No. 765068.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Automatic chord-scale recognition using harmonic pitch class profiles

  • Demirel, Emir
  • Bozkurt, Baris
  • Serra, Xavier
Comunicació presentada a: 16th Sound & Music Computing Conference celebrada del 23 al 31 de maig de 2019 a Málaga, Espanya., This study focuses on the application of different computational methods to carry out a ”modal harmonic analysis” for Jazz improvisation performances by modeling the concept of chord-scales. The Chord-Scale Theory is a theoretical concept that explains the relationship between the harmonic context of a musical piece and possible scale types to be used for improvisation. This work proposes different computational approaches for the recognition of the chordscale type in an improvised phrase given the harmonic context. We have curated a dataset to evaluate different chordscale recognition approaches proposed in this study, where the dataset consists of around 40 minutes of improvised monophonic Jazz solo performances. The dataset is made publicly available and shared on freesound.org. To achieve the task of chord-scale type recognition, we propose one rule-based, one probabilistic and one supervised learning method. All proposed methods use Harmonic Pitch Class Profile (HPCP) features for classification. We observed an increase in the classification score when learned chordscale models are filtered with predefined scale templates indicating that incorporating prior domain knowledge to learned models is beneficial. This study has its novelty in presenting a first computational analysis on chord-scales in the context of Jazz improvisation., This work is supported by ERC funded TECSOME project. The author E.D received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowsa-Curie grant agreement No. 765068.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Combining musical features for cover detection

  • Doras, Guillaume
  • Yesiler, Furkan
  • Serrà Julià, Joan
  • Gómez Gutiérrez, Emilia, 1975-
  • Peeters, Geoffroy
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Recent works have addressed the automatic cover detection problem from a metric learning perspective. They employ different input representations, aiming to exploit melodic or harmonic characteristics of songs and yield promising performances. In this work, we propose a comparative study of these different representations and show that systems combining melodic and harmonic features drastically outperform those relying on a single input representation. We illustrate how these features complement each other with both quantitative and qualitative analyses. We finally investigate various fusion schemes and propose methods yielding state-of-the-art performances on two publicly-available large datasets., FY is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068, and EG by TROMPA, the Horizon 2020 project 770376-2.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Neural percussive synthesis parameterised by high-level timbral features

  • Ramires, António
  • Chandna, Pritish
  • Favory, Xavier
  • Gómez Gutiérrez, Emilia, 1975-
  • Serra, Xavier
Comunicació presentada a: ICASSP 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, celebrat en línia del 4 al 8 de maig de 2020., We present a deep neural network-based methodology for synthesising percussive sounds with control over high-level timbral characteristics of the sounds. This approach allows for intuitive control of a synthesizer, enabling the user to shape sounds without extensive knowledge of signal processing. We use a feedforward convolutional neural network-based architecture, which is able to map input parameters to the corresponding waveform. We propose two datasets to evaluate our approach on both a restrictive context, and in one covering a broader spectrum of sounds. The timbral features used as parameters are taken from recent literature in signal processing. We also use these features for evaluation and validation of the presented model, to ensure that changing the input parameters produces a congruent waveform with the desired characteristics. Finally, we evaluate the quality of the output sound using a subjective listening test. We provide sound examples and the system's source code for reproducibility., This work is partially funded by the European Unions Horizon 2020 research and innovation programme under the Marie Skodowska-Curie grant agreement No765068, MIP-Frontiers. This work is partially supported by the Towards Richer Online Music Public-domain Archives (TROMPA) project. The TITANX used for this research was donated by the NVIDIA Corporation.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Less is more: faster and better music version identification with embedding distillation

  • Serrà Julià, Joan
  • Yesiler, Furkan
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Version identification systems aim to detect different renditions of the same underlying musical composition (loosely called cover songs). By learning to encode entire recordings into plain vector embeddings, recent systems have made significant progress in bridging the gap between accuracy and scalability, which has been a key challenge for nearly two decades. In this work, we propose to further narrow this gap by employing a set of data distillation techniques that reduce the embedding dimensionality of a pre-trained state-of-the-art model. We compare a wide range of techniques and propose new ones, from classical dimensionality reduction to more sophisticated distillation schemes. With those, we obtain 99% smaller embeddings that, moreover, yield up to a 3% accuracy increase. Such small embeddings can have an important impact in retrieval time, up to the point of making a real-world system practical on a standalone laptop., This work is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068, and by TROMPA, the Horizon 2020 project 770376-2.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Web interface for exploration of latent and tag spaces in music auto-tagging

  • Tovstogan, Philip
  • Serra, Xavier
  • Bogdanov, Dmitry
Presentat a Machine Learning for Media Discovery Workshop, celebrat dins The 37th International Conference on Machine Learning, We present an online graphical web interface for the exploration and evaluation of embedding and tag spaces of music auto-tagging systems. It allows for quick and qualitative evaluation of individual and pairwise tag predictions as well as visualization of tag and embedding latent spaces (original and with dimensionality reduction). We provide taggrams and embedding vectors for the MTG-Jamendo dataset from multiple state-of-theart auto-tagging models that can be explored and compared., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Da-TACOS: A dataset for cover song identification and understanding

  • Yesiler, Furkan
  • Tralie, Chris
  • Correya, Albin Andrew
  • Silva, Diego F.
  • Tovstogan, Philip
  • Gómez Gutiérrez, Emilia, 1975-
  • Serra, Xavier
Comunicació presentada a: 20th annual conference of the International Society for Music Information Retrieval (ISMIR) celebrat del 4 al 8 de novembre de 2019 a Delft, Països Baixos., This paper focuses on Cover Song Identification (CSI), an important research challenge in content-based Music Information Retrieval (MIR). Although the task itself is interesting and challenging for both academia and industry scenarios, there are a number of limitations for the advancement of current approaches. We specifically address two of them in the present study. First, the number of publicly available datasets for this task is limited, and there is no publicly available benchmark set that is widely used among researchers for comparative algorithm evaluation. Second, most of the algorithms are not publicly shared and reproducible, limiting the comparison of approaches. To overcome these limitations we propose Da-TACOS, a DaTAset for COver Song Identification and Understanding, and two frameworks for feature extraction and benchmarking to facilitate reproducibility. Da-TACOS contains 25K songs represented by unique editorial metadata plus 9 low- and mid-level features pre-computed with open source libraries, and is divided into two subsets. The Cover Analysis subset contains audio features (e.g. key, tempo) that can serve to study how musical characteristics vary for cover songs. The Benchmark subset contains the set of features that have been frequently used in CSI research, e.g. chroma, MFCC, beat onsets etc. Moreover, we provide initial benchmarking results of a selected number of state-of-the-art CSI algorithms using our dataset, and for reproducibility, we share a GitHub repository containing the feature extraction and benchmarking frameworks., This work is partially supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068, and by TROMPA, the Horizon 2020 project 770376-2.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Visualization of deep audio embeddings for music exploration and rediscovery

  • Tovstogan, Philip
  • Serra, Xavier
  • Bogdanov, Dmitry
Comunicació presentada a: 19th Sound and Music Computing Conference, celebrat del 5 al 12 de juny de 2022 a Sant-Étienne, User interfaces for music exploration and discovery have always been an exciting application of music information retrieval (MIR) throughout the years. However, while discovering new music is a common goal of such systems, there has been less attention paid to the exploration and rediscovery within personal music collections, where finding interesting relations between music items already familiar to the user can lead to a different type of highly engaging and rewarding experience. In this paper, we present a novel web interface to visualize music collections using the audio embeddings extracted from music tracks. The system allows exploring the relationship between music tracks from multiple perspectives, displaying embedding and tag spaces extracted by music auto-tagging models, trained using different architectures and datasets, coupled with various 2D projection algorithms. We conduct a user study to analyze the effectiveness of different visualization strategies on the participants’ personal music collections, particularly for playlist creation and music library navigation and rediscovery. Our results show that such an interface provides a good alternative to standard hierarchical library organization by metadata., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skøodowska-Curie grant agreement No. 765068.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

TIV.lib: An open-source library for the tonal description of musical audio

  • Ramires, António
  • Bernardes, Gilberto
  • Davies, Matthew E. P.
  • Serra, Xavier
Comunicació prsentada a: International Conference on Digital Audio Effects (DAFx) celebrada del 8 al 12 de setembre de 2020 a Viena, Àustria., In this paper, we present TIV.lib, an open-source library for the content-based tonal description of musical audio signals. Its main novelty relies on the perceptually-inspired Tonal Interval Vector space based on the Discrete Fourier transform, from which multiple instantaneous and global representations, descriptors and metrics are computed—e.g., harmonic change, dissonance, diatonicity, and musical key. The library is cross-platform, implemented in Python and the graphical programming language Pure Data, and can be used in both online and offline scenarios. Of note is its potential for enhanced Music Information Retrieval, where tonal descriptors sit at the core of numerous methods and applications., António Ramires is supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No765068, MIP-Frontiers. Gilberto Bernardes is supported by Experimentation in music in Portuguese culture: History, contexts, and practices in the 20th and 21st centuries—Project co-funded by the European Union, through the Operational Program Competitiveness and Internationalization, in its ERDF component, and by national funds, through the Portuguese Foundation for Science and Technology. This work is funded by national funds through the FCT - Foundation for Science and Technology, I.P., within the scope of the project CISUC - UID/CEC/00326/2020 and by European Social Fund, through the Regional Operational Program Centro 2020, as well as by Portuguese National Funds through the FCT - Foundation for Science and Technology, I.P., under the project IF/01566/2015.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Investigating the efficacy of music version retrieval systems for setlist identification

  • Yesiler, Furkan
  • Molina, Emilio
  • Serrà, Joan
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2021), celebrat del 6 a l'11 de juny de 2021 de manera virtual., The setlist identification (SLI) task addresses a music recognition use case where the goal is to retrieve the metadata and timestamps for all the tracks played in live music events. Due to various musical and non-musical changes in live performances, developing automatic SLI systems is still a challenging task that, despite its industrial relevance, has been under-explored in the academic literature. In this paper, we propose an end-to-end workflow that identifies relevant metadata and timestamps of live music performances using a version identification system. We compare 3 of such systems to investigate their suitability for this particular task. For developing and evaluating SLI systems, we also contribute a new dataset that contains 99.5 h of concerts with annotated metadata and timestamps, along with the corresponding reference set. The dataset is categorized by audio qualities and genres to analyze the performance of SLI systems in different use cases. Our approach can identify 68% of the annotated segments, with values ranging from 35% to 77% based on the genre. Finally, we evaluate our approach against a database of 56.8 k songs to illustrate the effect of expanding the reference set, where we can still identify 56% of the annotated segments., This work is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

MediaEval 2019: emotion and theme recognition in music using Jamendo

  • Bogdanov, Dmitry
  • Porter, Alastair
  • Tovstogan, Philip
  • Won, Minz
Comunicació presentada a: MediaEval 2019 Workshop celebrat del 27 al 30 d'octubre de 2019 a Sophia Antipolis, Fraança., This paper provides an overview of the Emotion and Theme recognition in Music task organized as part of the MediaEval 2019 Benchmarking Initiative for Multimedia Evaluation. The goal of this task is to automatically recognize the emotions and themes conveyed in a music recording by means of audio analysis. We provide a large dataset of audio and labels that the participants can use to train and evaluate their systems. We also provide a baseline solution that utilizes VGG-ish architecture. This overview paper presents the task challenges, the employed ground-truth information and dataset, and the evaluation methodology., This work has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 765068. This work was funded by the predoctoral grant MDM-2015-0502- 17-2 from the Spanish Ministry of Economy and Competitiveness linked to the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Improved automatic instrumentation role classification and loop activation transcription

  • Drysdale, Jake
  • Hockman, Jason
  • Ramires, António
  • Serra, Xavier
Comunicació presentada a 25th International Conference on Digital Audio Effects (DAFx20in22), celebrat del 6 al 10 de setembre de 2022 a Viena, Àustria., Many electronic music (EM) genres are composed through the activation of short audio recordings of instruments designed for seamless repetition—or loops. In this work, loops of key structural groups such as bass, percussive or melodic elements are labelled by the role they occupy in a piece of music through the task of automatic instrumentation role classification (AIRC). Such labels assist EM producers in the identification of compatible loops in large unstructured audio databases. While human annotation is often laborious, automatic classification allows for fast and scalable generation of these labels. We experiment with several deep-learning architectures and propose a data augmentation method for improving multi-label representation to balance classes within the Freesound Loop Dataset. To improve the classification accuracy of the architectures, we also evaluate different pooling operations. Results indicate that in combination with the data augmentation and pooling strategies, the proposed system achieves state-of-the-art performance for AIRC. Additionally, we demonstrate how our proposed AIRC method is useful for analysing the structure of EM compositions through loop activation transcription., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 765068, MIP-Frontiers.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

The MTG-Jamendo dataset for automatic music tagging

  • Bogdanov, Dmitry
  • Won, Minz
  • Tovstogan, Philip
  • Porter, Alastair
  • Serra, Xavier
Comunicació presentada a: ML4MD Machine Learning for Music Discovery Workshop del congrés ICML2019 celebrat el 15 de juny de 2019 a Long Beach, California., We present the MTG-Jamendo Dataset, a new open dataset for music auto-tagging. It is built using music available at Jamendo under Creative Commons licenses and tags provided by content uploaders. The dataset contains over 55,000 full audio tracks with 195 tags from genre, instrument, and mood/theme categories. We provide elaborated data splits for researchers and report the performance of a simple baseline approach on five different sets of tags: genre, instrument, mood/theme, top-50, and overall., This work was funded by the predoctoral grant MDM-2015- 0502-17-2 from the Spanish Ministry of Economy and Competitiveness linked to the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502). This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068. This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 688382 “AudioCommons”.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

A study of control methods for percussive sound synthesis based on gans

  • Ramires, António
  • Juras, Jordan
  • Parker, Julian D.
  • Serra, Xavier
Comunicació presentada a 25th International Conference on Digital Audio Effects (DAFx20in22), celebrat del 6 al 10 de setembre de 2022 a Viena, Àustria., The process of creating drum sounds has seen significant evolution in the past decades. The development of analogue drum synthesizers, such as the TR-808, and modern sound design tools in Digital Audio Workstations led to a variety of drum timbres that defined entire musical genres. Recently, drum synthesis research has been revived with a new focus on training generative neural networks to create drum sounds. Different interfaces have previously been proposed to control the generative process, from low-level latent space navigation to high-level semantic feature parameterisation, but no comprehensive analysis has been presented to evaluate how each approach relates to the creative process. We aim to evaluate how different interfaces support creative control over drum generation by conducting a user study based on the Creative Support Index. We experiment with both a supervised method that decodes semantic latent space directions and an unsupervised Closed-Form Factorization approach from computer vision literature to parameterise the generation process and demonstrate that the latter is the preferred means to control a drum synthesizer based on the StyleGAN2 network architecture., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No 765068, MIP-Frontiers.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Assessing algorithmic biases for musical version identification

  • Yesiler, Furkan
  • Serrà, Joan
  • Miron, Marius
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a 15th ACM International Conference on Web Search and Data Mining (WSDM 2022), celebrat del 21 al 25 de febrer de 2022 de manera virtual., Version identification (VI) systems now offer accurate and scalable solutions for detecting different renditions of a musical composition, allowing the use of these systems in industrial applications and throughout the wider music ecosystem. Such use can have an important impact on various stakeholders regarding recognition and financial benefits, including how royalties are circulated for digital rights management. In this work, we take a step toward acknowledging this impact and consider VI systems as socio-technical systems rather than isolated technologies. We propose a framework for quantifying performance disparities across 5 systems and 6 relevant side attributes: gender, popularity, country, language, year, and prevalence. We also consider 3 main stakeholders for this particular information retrieval use case: the performing artists of query tracks, those of reference (original) tracks, and the composers. By categorizing the recordings in our dataset using such attributes and stakeholders, we analyze whether the considered VI systems show any implicit biases. We find signs of disparities in identification performance for most of the groups we include in our analyses. We also find that learning- and rule-based systems behave differently for some attributes, which suggests an additional dimension to consider along with accuracy and scalability when evaluating VI systems. Lastly, we share our dataset to encourage VI researchers to take these aspects into account while building new systems., This work is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Visualization of deep audio embeddings for music exploration and rediscovery

  • Tovstogan, Philip
  • Serra, Xavier
  • Bogdanov, Dmitry
Comunicació presentada a: 19th Sound and Music Computing Conference, celebrat del 5 al 12 de juny de 2022 a Sant-Étienne, User interfaces for music exploration and discovery have always been an exciting application of music information retrieval (MIR) throughout the years. However, while discovering new music is a common goal of such systems, there has been less attention paid to the exploration and rediscovery within personal music collections, where finding interesting relations between music items already familiar to the user can lead to a different type of highly engaging and rewarding experience. In this paper, we present a novel web interface to visualize music collections using the audio embeddings extracted from music tracks. The system allows exploring the relationship between music tracks from multiple perspectives, displaying embedding and tag spaces extracted by music auto-tagging models, trained using different architectures and datasets, coupled with various 2D projection algorithms. We conduct a user study to analyze the effectiveness of different visualization strategies on the participants’ personal music collections, particularly for playlist creation and music library navigation and rediscovery. Our results show that such an interface provides a good alternative to standard hierarchical library organization by metadata., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skøodowska-Curie grant agreement No. 765068.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Similarity of nearest-neighbor query results in deep latent spaces

  • Tovstogan, Philip
  • Serra, Xavier
  • Bogdanov, Dmitry
Comunicació presentada a: 19th Sound and Music Computing Conference, celebrat del 5 al 12 de juny de 2022 a Sant-Étienne, Music recommendation systems are commonly used for personalized recommendations. However, there are cases where due to privacy concerns or design decisions, there is no user information nor collaborative filtering data available. In those cases, it is possible to use content-based similarity spaces to retrieve the most similar tracks to be recommended based on the reference track. In this paper, we compare the latent spaces extracted from state-of-the-art autotagging models in terms of the similarity between lists of retrieved nearest neighbors. We additionally study item factors from collaborative-filtering data as a reference. We provide insights into how much the choice of the architecture, training dataset, or model layer (output vs. penultimate) as well as a projection of the latent space onto 2D changes the list of retrieved nearest neighbors. We release the dataset of 9 content-based and 3 collaborative-filtering latent representations of 29 275 tracks from Jamendo that we use for the evaluation. Moreover, we perform an online user experiment to compare the perceived track-to-track similarity of the selected evaluated latent spaces. The results show that content-based spaces show better results in our scenario, particularly embeddings from penultimate layers of auto-tagging architectures., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skøodowska-Curie grant agreement No. 765068.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Automatic chord-scale recognition using harmonic pitch class profiles

  • Demirel, Emir
  • Bozkurt, Baris
  • Serra, Xavier
Comunicació presentada a: 16th Sound & Music Computing Conference celebrada del 23 al 31 de maig de 2019 a Málaga, Espanya., This study focuses on the application of different computational methods to carry out a ”modal harmonic analysis” for Jazz improvisation performances by modeling the concept of chord-scales. The Chord-Scale Theory is a theoretical concept that explains the relationship between the harmonic context of a musical piece and possible scale types to be used for improvisation. This work proposes different computational approaches for the recognition of the chordscale type in an improvised phrase given the harmonic context. We have curated a dataset to evaluate different chordscale recognition approaches proposed in this study, where the dataset consists of around 40 minutes of improvised monophonic Jazz solo performances. The dataset is made publicly available and shared on freesound.org. To achieve the task of chord-scale type recognition, we propose one rule-based, one probabilistic and one supervised learning method. All proposed methods use Harmonic Pitch Class Profile (HPCP) features for classification. We observed an increase in the classification score when learned chordscale models are filtered with predefined scale templates indicating that incorporating prior domain knowledge to learned models is beneficial. This study has its novelty in presenting a first computational analysis on chord-scales in the context of Jazz improvisation., This work is supported by ERC funded TECSOME project. The author E.D received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowsa-Curie grant agreement No. 765068.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

The MTG-Jamendo dataset for automatic music tagging

  • Bogdanov, Dmitry
  • Won, Minz
  • Tovstogan, Philip
  • Porter, Alastair
  • Serra, Xavier
Comunicació presentada a: ML4MD Machine Learning for Music Discovery Workshop del congrés ICML2019 celebrat el 15 de juny de 2019 a Long Beach, California., We present the MTG-Jamendo Dataset, a new open dataset for music auto-tagging. It is built using music available at Jamendo under Creative Commons licenses and tags provided by content uploaders. The dataset contains over 55,000 full audio tracks with 195 tags from genre, instrument, and mood/theme categories. We provide elaborated data splits for researchers and report the performance of a simple baseline approach on five different sets of tags: genre, instrument, mood/theme, top-50, and overall., This work was funded by the predoctoral grant MDM-2015- 0502-17-2 from the Spanish Ministry of Economy and Competitiveness linked to the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502). This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068. This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 688382 “AudioCommons”.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Neural percussive synthesis parameterised by high-level timbral features

  • Ramires, António
  • Chandna, Pritish
  • Favory, Xavier
  • Gómez Gutiérrez, Emilia, 1975-
  • Serra, Xavier
Comunicació presentada a: ICASSP 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, celebrat en línia del 4 al 8 de maig de 2020., We present a deep neural network-based methodology for synthesising percussive sounds with control over high-level timbral characteristics of the sounds. This approach allows for intuitive control of a synthesizer, enabling the user to shape sounds without extensive knowledge of signal processing. We use a feedforward convolutional neural network-based architecture, which is able to map input parameters to the corresponding waveform. We propose two datasets to evaluate our approach on both a restrictive context, and in one covering a broader spectrum of sounds. The timbral features used as parameters are taken from recent literature in signal processing. We also use these features for evaluation and validation of the presented model, to ensure that changing the input parameters produces a congruent waveform with the desired characteristics. Finally, we evaluate the quality of the output sound using a subjective listening test. We provide sound examples and the system's source code for reproducibility., This work is partially funded by the European Unions Horizon 2020 research and innovation programme under the Marie Skodowska-Curie grant agreement No765068, MIP-Frontiers. This work is partially supported by the Towards Richer Online Music Public-domain Archives (TROMPA) project. The TITANX used for this research was donated by the NVIDIA Corporation.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Investigating the efficacy of music version retrieval systems for setlist identification

  • Yesiler, Furkan
  • Molina, Emilio
  • Serrà, Joan
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2021), celebrat del 6 a l'11 de juny de 2021 de manera virtual., The setlist identification (SLI) task addresses a music recognition use case where the goal is to retrieve the metadata and timestamps for all the tracks played in live music events. Due to various musical and non-musical changes in live performances, developing automatic SLI systems is still a challenging task that, despite its industrial relevance, has been under-explored in the academic literature. In this paper, we propose an end-to-end workflow that identifies relevant metadata and timestamps of live music performances using a version identification system. We compare 3 of such systems to investigate their suitability for this particular task. For developing and evaluating SLI systems, we also contribute a new dataset that contains 99.5 h of concerts with annotated metadata and timestamps, along with the corresponding reference set. The dataset is categorized by audio qualities and genres to analyze the performance of SLI systems in different use cases. Our approach can identify 68% of the annotated segments, with values ranging from 35% to 77% based on the genre. Finally, we evaluate our approach against a database of 56.8 k songs to illustrate the effect of expanding the reference set, where we can still identify 56% of the annotated segments., This work is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

MediaEval 2019: emotion and theme recognition in music using Jamendo

  • Bogdanov, Dmitry
  • Porter, Alastair
  • Tovstogan, Philip
  • Won, Minz
Comunicació presentada a: MediaEval 2019 Workshop celebrat del 27 al 30 d'octubre de 2019 a Sophia Antipolis, Fraança., This paper provides an overview of the Emotion and Theme recognition in Music task organized as part of the MediaEval 2019 Benchmarking Initiative for Multimedia Evaluation. The goal of this task is to automatically recognize the emotions and themes conveyed in a music recording by means of audio analysis. We provide a large dataset of audio and labels that the participants can use to train and evaluate their systems. We also provide a baseline solution that utilizes VGG-ish architecture. This overview paper presents the task challenges, the employed ground-truth information and dataset, and the evaluation methodology., This work has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 765068. This work was funded by the predoctoral grant MDM-2015-0502- 17-2 from the Spanish Ministry of Economy and Competitiveness linked to the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Accurate and scalable version identification using musically-motivated embeddings

  • Yesiler, Furkan
  • Serrà Julià, Joan
  • Gómez Gutiérrez, Emilia, 1975-
The version identification (VI) task deals with the automatic detection of recordings that correspond to the same underlying musical piece. Despite many efforts, VI is still an open problem, with much room for improvement, specially with regard to combining accuracy and scalability. In this paper, we present MOVE, a musically-motivated method for accurate and scalable version identification. MOVE achieves state-of-the-art performance on two publicly-available benchmark sets by learning scalable embeddings in an Euclidean distance space, using a triplet loss and a hard triplet mining strategy. It improves over previous work by employing an alternative input representation, and introducing a novel technique for temporal content summarization, a standardized latent space, and a data augmentation strategy specifically designed for VI. In addition to the main results, we perform an ablation study to highlight the importance of our design choices, and study the relation between embedding dimensionality and model performance., Comunicació presentada a: ICASSP 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, celebrat en línia del 4 al 8 de maig de 2020., This work is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skodowska-Curie grant agreement No. 765068, and by TROMPA, the Horizon 2020 project 770376-2.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Web interface for exploration of latent and tag spaces in music auto-tagging

  • Tovstogan, Philip
  • Serra, Xavier
  • Bogdanov, Dmitry
Presentat a Machine Learning for Media Discovery Workshop, celebrat dins The 37th International Conference on Machine Learning, We present an online graphical web interface for the exploration and evaluation of embedding and tag spaces of music auto-tagging systems. It allows for quick and qualitative evaluation of individual and pairwise tag predictions as well as visualization of tag and embedding latent spaces (original and with dimensionality reduction). We provide taggrams and embedding vectors for the MTG-Jamendo dataset from multiple state-of-theart auto-tagging models that can be explored and compared., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Da-TACOS: A dataset for cover song identification and understanding

  • Yesiler, Furkan
  • Tralie, Chris
  • Correya, Albin Andrew
  • Silva, Diego F.
  • Tovstogan, Philip
  • Gómez Gutiérrez, Emilia, 1975-
  • Serra, Xavier
Comunicació presentada a: 20th annual conference of the International Society for Music Information Retrieval (ISMIR) celebrat del 4 al 8 de novembre de 2019 a Delft, Països Baixos., This paper focuses on Cover Song Identification (CSI), an important research challenge in content-based Music Information Retrieval (MIR). Although the task itself is interesting and challenging for both academia and industry scenarios, there are a number of limitations for the advancement of current approaches. We specifically address two of them in the present study. First, the number of publicly available datasets for this task is limited, and there is no publicly available benchmark set that is widely used among researchers for comparative algorithm evaluation. Second, most of the algorithms are not publicly shared and reproducible, limiting the comparison of approaches. To overcome these limitations we propose Da-TACOS, a DaTAset for COver Song Identification and Understanding, and two frameworks for feature extraction and benchmarking to facilitate reproducibility. Da-TACOS contains 25K songs represented by unique editorial metadata plus 9 low- and mid-level features pre-computed with open source libraries, and is divided into two subsets. The Cover Analysis subset contains audio features (e.g. key, tempo) that can serve to study how musical characteristics vary for cover songs. The Benchmark subset contains the set of features that have been frequently used in CSI research, e.g. chroma, MFCC, beat onsets etc. Moreover, we provide initial benchmarking results of a selected number of state-of-the-art CSI algorithms using our dataset, and for reproducibility, we share a GitHub repository containing the feature extraction and benchmarking frameworks., This work is partially supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068, and by TROMPA, the Horizon 2020 project 770376-2.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A study of control methods for percussive sound synthesis based on gans

  • Ramires, António
  • Juras, Jordan
  • Parker, Julian D.
  • Serra, Xavier
Comunicació presentada a 25th International Conference on Digital Audio Effects (DAFx20in22), celebrat del 6 al 10 de setembre de 2022 a Viena, Àustria., The process of creating drum sounds has seen significant evolution in the past decades. The development of analogue drum synthesizers, such as the TR-808, and modern sound design tools in Digital Audio Workstations led to a variety of drum timbres that defined entire musical genres. Recently, drum synthesis research has been revived with a new focus on training generative neural networks to create drum sounds. Different interfaces have previously been proposed to control the generative process, from low-level latent space navigation to high-level semantic feature parameterisation, but no comprehensive analysis has been presented to evaluate how each approach relates to the creative process. We aim to evaluate how different interfaces support creative control over drum generation by conducting a user study based on the Creative Support Index. We experiment with both a supervised method that decodes semantic latent space directions and an unsupervised Closed-Form Factorization approach from computer vision literature to parameterise the generation process and demonstrate that the latter is the preferred means to control a drum synthesizer based on the StyleGAN2 network architecture., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No 765068, MIP-Frontiers.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Improved automatic instrumentation role classification and loop activation transcription

  • Drysdale, Jake
  • Hockman, Jason
  • Ramires, António
  • Serra, Xavier
Comunicació presentada a 25th International Conference on Digital Audio Effects (DAFx20in22), celebrat del 6 al 10 de setembre de 2022 a Viena, Àustria., Many electronic music (EM) genres are composed through the activation of short audio recordings of instruments designed for seamless repetition—or loops. In this work, loops of key structural groups such as bass, percussive or melodic elements are labelled by the role they occupy in a piece of music through the task of automatic instrumentation role classification (AIRC). Such labels assist EM producers in the identification of compatible loops in large unstructured audio databases. While human annotation is often laborious, automatic classification allows for fast and scalable generation of these labels. We experiment with several deep-learning architectures and propose a data augmentation method for improving multi-label representation to balance classes within the Freesound Loop Dataset. To improve the classification accuracy of the architectures, we also evaluate different pooling operations. Results indicate that in combination with the data augmentation and pooling strategies, the proposed system achieves state-of-the-art performance for AIRC. Additionally, we demonstrate how our proposed AIRC method is useful for analysing the structure of EM compositions through loop activation transcription., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 765068, MIP-Frontiers.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

The Freesound Loop dataset and annotation tool

  • Ramires, António
  • Font Corbera, Frederic
  • Bogdanov, Dmitry
  • Smith, Jordan B. L.
  • Yang, Yi-Hsuan
  • Ching, Joann
  • Chen, Bo-Yu
  • Wu, Yueh-Kao
  • Wei-Han, Hsu
  • Serra, Xavier
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Music loops are essential ingredients in electronic music production, and there is a high demand for pre-recorded loops in a variety of styles. Several commercial and community databases have been created to meet this demand, but most are not suitable for research due to their strict licensing. We present the Freesound Loop Dataset (FSLD), a new large-scale dataset of music loops annotated by experts. The loops originate from Freesound, a community database of audio recordings released under Creative Commons licenses, so the audio in our dataset may be redistributed. The annotations include instrument, tempo, meter, key and genre tags. We describe the methodology used to assemble and annotate the data, and report on the distribution of tags in the data and inter-annotator agreement. We also present to the community an online loop annotator tool that we developed. To illustrate the usefulness of FSLD, we present short case studies on using it to estimate tempo and key, generate music tracks, and evaluate a loop separation algorithm. We anticipate that the community will find yet more uses for the data, in applications from automatic loop characterisation to algorithmic composition., This research was funded in part by European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No765068, MIP-Frontiers and by a grant from the Ministry of Science and Technology, Taiwan (MOST107-2221-E-001-013-MY2).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

TIV.lib: An open-source library for the tonal description of musical audio

  • Ramires, António
  • Bernardes, Gilberto
  • Davies, Matthew E. P.
  • Serra, Xavier
Comunicació prsentada a: International Conference on Digital Audio Effects (DAFx) celebrada del 8 al 12 de setembre de 2020 a Viena, Àustria., In this paper, we present TIV.lib, an open-source library for the content-based tonal description of musical audio signals. Its main novelty relies on the perceptually-inspired Tonal Interval Vector space based on the Discrete Fourier transform, from which multiple instantaneous and global representations, descriptors and metrics are computed—e.g., harmonic change, dissonance, diatonicity, and musical key. The library is cross-platform, implemented in Python and the graphical programming language Pure Data, and can be used in both online and offline scenarios. Of note is its potential for enhanced Music Information Retrieval, where tonal descriptors sit at the core of numerous methods and applications., António Ramires is supported by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No765068, MIP-Frontiers. Gilberto Bernardes is supported by Experimentation in music in Portuguese culture: History, contexts, and practices in the 20th and 21st centuries—Project co-funded by the European Union, through the Operational Program Competitiveness and Internationalization, in its ERDF component, and by national funds, through the Portuguese Foundation for Science and Technology. This work is funded by national funds through the FCT - Foundation for Science and Technology, I.P., within the scope of the project CISUC - UID/CEC/00326/2020 and by European Social Fund, through the Regional Operational Program Centro 2020, as well as by Portuguese National Funds through the FCT - Foundation for Science and Technology, I.P., under the project IF/01566/2015.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Less is more: faster and better music version identification with embedding distillation

  • Serrà Julià, Joan
  • Yesiler, Furkan
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Version identification systems aim to detect different renditions of the same underlying musical composition (loosely called cover songs). By learning to encode entire recordings into plain vector embeddings, recent systems have made significant progress in bridging the gap between accuracy and scalability, which has been a key challenge for nearly two decades. In this work, we propose to further narrow this gap by employing a set of data distillation techniques that reduce the embedding dimensionality of a pre-trained state-of-the-art model. We compare a wide range of techniques and propose new ones, from classical dimensionality reduction to more sophisticated distillation schemes. With those, we obtain 99% smaller embeddings that, moreover, yield up to a 3% accuracy increase. Such small embeddings can have an important impact in retrieval time, up to the point of making a real-world system practical on a standalone laptop., This work is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068, and by TROMPA, the Horizon 2020 project 770376-2.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Combining musical features for cover detection

  • Doras, Guillaume
  • Yesiler, Furkan
  • Serrà Julià, Joan
  • Gómez Gutiérrez, Emilia, 1975-
  • Peeters, Geoffroy
Comunicació presentada a: International Society for Music Information Retrieval Conference celebrat de l'11 al 16 d'octubre de 2020 de manera virtual., Recent works have addressed the automatic cover detection problem from a metric learning perspective. They employ different input representations, aiming to exploit melodic or harmonic characteristics of songs and yield promising performances. In this work, we propose a comparative study of these different representations and show that systems combining melodic and harmonic features drastically outperform those relying on a single input representation. We illustrate how these features complement each other with both quantitative and qualitative analyses. We finally investigate various fusion schemes and propose methods yielding state-of-the-art performances on two publicly-available large datasets., FY is supported by the MIP-Frontiers project, the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068, and EG by TROMPA, the Horizon 2020 project 770376-2.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Data augmentation for instrument classification robust to audio effects

  • Ramires, António
  • Serra, Xavier
Comunicació presentada a la 22a International Conference on Digital Audio Effects (DAFx-19) que se celebra del 2 al 6 de setembre de 2019 a Birmingham, Regne Unit., Reusing recorded sounds (sampling) is a key component in Electronic Music Production (EMP), which has been present since its early days and is at the core of genres like hip-hop or jungle. Commercial and non-commercial services allow users to obtain collections of sounds (sample packs) to reuse in their compositions. Automatic classification of one-shot instrumental sounds allows automatically categorising the sounds contained in these collections, allowing easier navigation and better characterisation. Automatic instrument classification has mostly targeted the classification of unprocessed isolated instrumental sounds or detecting predominant instruments in mixed music tracks. For this classification to be useful in audio databases for EMP, it has to be robust to the audio effects applied to unprocessed sounds. In this paper we evaluate how a state of the art model trained with a large dataset of one-shot instrumental sounds performs when classifying instruments processed with audio effects. In order to evaluate the robustness of the model, we use data augmentation with audio effects and evaluate how each effect influences the classification accuracy., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 765068, MIP-Frontiers. We thank Matthew Davies for reviewing a draft of this paper and providing helpful feedback.