Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 222
Encontrada(s) 3 página(s)

Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Phrase-based rāga recognition using vector space modeling

  • Gulati, Sankalp
  • Serrà Julià, Joan
  • Ishwar, Vignesh
  • Sentürk, Sertan
  • Serra, Xavier
Comunicació presentada a la 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), celebrada els dies 20 a 25 de març a Xangai, Xina., Automatic raga recognition is one of the fundamental computational tasks in Indian art music. Motivated by the way seasoned listeners identify ragas, we propose a raga recognition approach based on melodic phrases. Firstly, we extract melodic patterns from a collection of audio recordings in an unsupervised way. Next, we group similar patterns by exploiting complex networks concepts and techniques. Drawing an analogy to topic modeling in text classification, we then represent audio recordings using a vector space model. Finally, we employ a number of classification strategies to build a predictive model for raga recognition. To evaluate our approach, we compile a music collection of over 124 hours, comprising 480 recordings and 40 ragas. We obtain 70% accuracy with the full 40-raga collection, and up to 92% accuracy with its 10-raga subset. We show that phrase-based raga recognition is a successful strategy, on par with the state of the art, and sometimes outperforms it. A by-product of our approach, which arguably is as important as the task of raga recognition, is the identification of raga-phrases. These phrases can be used as a dictionary of semantically-meaningful melodic units for several computational tasks in Indian art music., This work is partly supported by the European Research Council under the European Unions Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Data-driven exploration of melodic structure in Hindustani music

  • Ganguli, Kaustuv Kanti
  • Gulati, Sankalp
  • Serra, Xavier
  • Rao, Preeti
Comunicació presentada a la 17th International Society for Music Information Retrieval Conference (ISMIR 2016), celebrada els dies 7 a 11 d'agost de 2016 a Nova York, EUA., Indian art music is quintessentially an improvisatory music form in which the line between ‘fixed’ and ‘free’ is extremely subtle. In a raga performance, the melody is loosely constrained by the chosen composition but otherwise improvised in accordance with the raga grammar. One of the melodic aspects that is governed by this grammar is the manner in which a melody evolves in time in the course of a performance. In this work, we aim to discover such implicit patterns or regularities present in the temporal evolution of vocal melodies of Hindustani music. We start by applying existing tools and techniques used in music information retrieval to a collection of concerts recordings of alap performances by renowned khayal vocal artists. We use svara-based and svara duration-based melodic features to study and quantify the manifestation of concepts such as vadi, samvadi, nyas and graha svara in the vocal performances. We show that the discovered patterns corroborate the musicological findings that describe the “unfolding” of a raga in vocal performances of Hindustani music. The patterns discovered from the vocal melodies might help music students to learn improvisation and can complement the oral music pedagogy followed in this music tradition., This work received partial funding from the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013)/ERC grant agreement 267583 (CompMusic). Part of the work was supported by Bharti Centre for Communication in IIT, Bombay.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Freesound 2: An improved platform for sharing audio clips

  • Akkermans, Vincent
  • Font Corbera, Frederic
  • Funollet, Jordi
  • de Jong, Bram
  • Roma Trepat, Gerard
  • Togias, Stelios
  • Serra, Xavier
Freesound.org is an online collaborative sound database where people from different disciplines share recorded sound clips under Creative Commons licenses. It was started in 2005 and it is being further developed by the Music Technology Group (MTG) of the Universitat Pompeu Fabra. Freesound’s initial goal was to give support to sound researchers, who often have trouble finding large royalty-free sound databases to test their algorithms, and to sound artists, who use pre-recorded sounds in their pieces. After six years since its inception, Freesound has become one of the most popular sites for sharing sound snippets. It serves around 30,000 unique visits per day and has more than two million registered users accessing the more than 120,000 uploaded sounds. Perhaps more importantly, there is a highly engaged community of users continuously contributing to the site, not only uploading sounds but also commenting, rating and discussing in the forums about relevant topics for the community. This remarkable growth surpassed our expectations and forced us to redesign and rebuild Freesound from scratch. This demo shows the new version of Freesound which brings a number of improvements and new functionalities. Some of these improvements facilitate the access to the content and provide a new framework for developers and researchers that are interested in working with Freesound. What follows is a summarized description of the system and the types of content that can be found.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Creating an a cappella singing audio dataset for automatic Jingju singing evaluation research

  • Gong, Rong
  • Caro Repetto, Rafael
  • Serra, Xavier
Comunicació presentada al 4th International Workshop on Digital Libraries for Musicology celebrat el 28 d'octubre de 2017 a Shanghai, Xina., e data-driven computational research on automatic jingju (also known as Beijing or Peking opera) singing evaluation lacks a suitable and comprehensive a cappella singing audio dataset. In this work, we present an a cappella singing audio dataset which consists of 120 arias, accounting for 1265 melodic lines. is dataset is also an extension our existing CompMusic jingju corpus. Both professional and amateur singers were invited to the dataset recording sessions, and the most common jingju musical elements have been covered. is dataset is also accompanied by metadata per aria and melodic line annotated for automatic singing evaluation research purpose. All the gathered data is openly available online., This research was funded by the European Research Council under the European Union's Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

AcousticBrainz: a community platform for gathering music information obtained from audio

  • Porter, Alastair
  • Bogdanov, Dmitry
  • Kaye, Robert
  • Tsukanov, Roman
  • Serra, Xavier
Comunicació presentada a la 16th International Society for Music Information Retrieval Conference (ISMIR 2015), celebrada els dies 26 a 30 d'octubre de 2015 a Màlaga, Espanya., We introduce the AcousticBrainz project, an open platform for gathering music information. At its core, AcousticBrainz is a database of music descriptors computed from audio recordings using a number of state-of-the-art Music Information Retrieval algorithms. Users run a supplied feature extractor on audio files and upload the analysis results to the AcousticBrainz server. All submissions include a MusicBrainz identifier allowing them to be linked to various sources of editorial information. The feature extractor is based on the open source Essentia audio analysis library. From the data submitted by the community, we run classifiers aimed at adding musically relevant semantic information. These classifiers can be developed by the community using tools available on the AcousticBrainz website. All data in AcousticBrainz is freely available and can be accessed through the website or API. For AcousticBrainz to be successful we need to have an active community that contributes to and uses this platform, and it is this community that will define the actual uses and applications of its data., This research has been partially funded by the CompMusic (ERC 267583) and SIGMUS (TIN2012-36650) projects.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Lyrics to audio alignment in polyphonic audio

  • Dzhambazov, Georgi Bogomilov
  • Miron, Marius
  • Serra, Xavier
Comunicació presentada a la 17th International Society for Music Information Retrieval Conference (ISMIR 2016), celebrada els dies 7 a 11 d'agost de 2018 a Nova York, EUA., In this paper we describe the two algorithms we submitted for the MIREX 2017 task of Automatic Lyrics-to-Audio Alignment. The task has as a goal the automatic detection of word boundaries in multi-instrumental English pop music. We rely on a phonetic recognizer based on hidden Markov models (HMM): a widely-used method for tracking phonemes in speech processing problems. Tracking lyrics in music audio is harder than tracking text in speech because, unlike speech, the singing voice is mixed with multiple instruments. To address this obstacle we propose the application of two separate methods for segregating the singing voice from the multi-instrumental mix. One of them is based on the detection of vocal harmonic partials, whereas the other extracts the vocal content by means of source separation., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583) and partly by the Spanish Ministry of Economy and Competitiveness, through the ”María de Maeztu” Programme for Centres/Units of Excellence in R&D” (MDM-2015-0502).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Melodic transformation processes in the arrangements of Jingju Banshi

  • Caro Repetto, Rafael
  • Serra, Xavier
Comunicació presentada a: Fourth International Conference On Analytical Approaches To World Music celebrat del 8 a l'11 de juny de 2016 a Nova York, Estats Units., Music in jingju (also known as Peking or Beijing opera) is predefined by tradition. Before the appearance of jingju music composers, actors used to “arrange melodies” (bianqu) for new lyrics according to melodic systems called shengqiang. Each shengqiang consists in a distinctive melodic framework which is transformed rhythmically into predefined metrical patterns called banshi to convey different emotions. If this is common knowledge in musicological literature and among performers, an analysis of how this melodic material is transformed is still to be undertaken. In this paper we present a preliminary approach to this topic, by implementing a computer aided comparative analysis. To this aim, we focus on three banshi in the xipi shengqiang as sang by the dan role-type, namely yuanban, which is considered to convey the “original” melody, manban, obtained by stretching yuanban, and kuaiban, a compression of yuanban. In order to ensure representativeness, we have gathered those arias quoted as example in several jingju music textbooks to build our dataset. We obtain a representation of the underlying melodic structure by comparing yuanban arias, search for trends in its transformation processes towards manban and kuaiban, and complement these results with statistical information computed from scores using the Music21 toolkit.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Singing voice phoneme segmentation by hierarchically inferring syllable and phoneme onset positions

  • Gong, Rong
  • Serra, Xavier
Comunicació presentada a: Interspeech 2018, celebrada del 2 al 6 de setembre de 2018 a Hyderabad, India., In this paper, we tackle the singing voice phoneme segmentation problem in the singing training scenario by using language independent information – onset and prior coarse duration. We propose a two-step method. In the first step, we jointly calculate the syllable and phoneme onset detection functions (ODFs) using a convolutional neural network (CNN). In the second step, the syllable and phoneme boundaries and labels are inferred hierarchically by using a duration-informed hidden Markov model (HMM). To achieve the inference, we incorporate the a priori duration model as the transition probabilities and the ODFs as the emission probabilities into the HMM. The proposed method is designed in a language-independent way such that no phoneme class labels are used. For the model training and algorithm evaluation, we collect a new jingju (also known as Beijing or Peking opera) solo singing voice dataset and manually annotate the boundaries and labels at phrase, syllable and phoneme levels. The dataset is publicly available. The proposed method is compared with a baseline method based on hidden semi-Markov model (HSMM) forced alignment. The evaluation results show that the proposed method outperforms the baseline by a large margin regarding both segmentation and onset detection tasks., This work is supported by the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A supervised approach to hierarchical metrical cycle tracking from audio music recordings

  • Srinivasamurthy, Ajay
  • Serra, Xavier
A supervised approach to metrical cycle tracking from audio is presented,/nwith a main focus on tracking the tāḷa, the hierarchical cyclic/nmetrical structure in Carnatic music. Given the tāḷa of a piece, we aim/nto estimate the akṣara (lowest metrical pulse), the akṣara period, and/nthe sama (first pulse of the tāḷa cycle). Starting with percussion enhanced/naudio, we estimate the akṣara pulse period from a tempogram/ncomputed using an onset detection function. A novelty function is/ncomputed using a self similarity matrix constructed using frame level/naudio features. These are then used to estimate possible akṣara and/nsama candidates, followed by a candidate selection based on periodicity/nconstraints, which leads to the final estimates. The algorithm/nis tested on an annotated collection of 176 pieces spanning four different/ntāḷas. Though applied to Carnatic music, the framework presented/nis general and can be extended to other music cultures with/ncyclical metrical structures., This work is partly supported by the European Research Council under/nthe European Union’s Seventh Framework Program, as part of the CompMusic/nproject (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A Musically aware system for browsing and interacting with audio music collections

  • Sordo, Mohamed
  • Koduri, Gopala Krishna
  • Sentürk, Sertan
  • Gulati, Sankalp
  • Serra, Xavier
Comunicació presentada al 2nd CompMusic Workshop, celebrat els dies 12 i 13 de juliol de 2012 a Istanbul (Turquia), organitzat per CompMusic., In the context of the CompMusic project we are developing methods to automatically describe/annotate audio music recordings pertaining to various music cultures. As a/nway to demonstrate the usefulness of the methods we are also developing a system to browse and interact with specific audio collections. The system is an online web application that interfaces with all the data gathered (audio, scores plus contextual information) and all the descriptions that are automatically generated with the developed methods. In this paper we present the basic architecture of the proposed system, the types of data sources that it includes,/nand we mention some of the culture specific issues that we are working on for its development. The system is in a preliminary stage but it shows the potential that MIR technologies can have in browsing and interacting with music/ncollections of various cultures., The CompMusic project is funded by the European Research Council under the European Unions Seventh Framework Programme (FP7/2007-2013) / ERC grant agreement 267583.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

An evaluation of methodologies for melodic similarity in audio recordings of Indian art music

  • Gulati, Sankalp
  • Serrà Julià, Joan
  • Serra, Xavier
Comunicació presentada a l'ICASSP 2015, International Conference on Acoustics, Speech, and Signal Processing, que es va celebrar els dies 19 al 24 d'abril de 2015 a Brisbane, Austràlia., We perform a comparative evaluation of methodologies for computing similarity between short-time melodic fragments of audio recordings of Indian art music. We experiment with 560 different combinations of procedures and parameter values. These include the choices made for the sampling rate of the melody representation, pitch quantization levels, normalization techniques and distance measures. The dataset used for evaluation consists of 157 and 340 annotated melodic fragments of Carnatic and Hindustani music recordings, respectively. Our results indicate that melodic fragment similarity is particularly sensitive to distance measures and normalization techniques. Sampling rates do not have a significant impact for Hindustani music, but can significantly degrade the performance for Carnatic music. Overall, the performed evaluation provides a better understanding of the processing steps and parameter settings for melodic similarity in Indian art music. Importantly, it paves the way for developing unsupervised melodic pattern discovery approaches, whose evaluation is a challenging and, many times, ill-defined task., This work is partly supported by the European Research Council under the European Union's Seventh Framework Program, as part of the Comp-Music project (ERC grant agreement 267583). JS acknowledges 2009-SGR-1434 from Generalitat de Catalunya and ICT-2011-8-318770 from the European Commission.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A knowledge based signal processing approach to tonic identification in indian classical music

  • Bellur, Ashwin
  • Ishwar, Vignesh
  • Serra, Xavier
  • Murthy, Hema A.
In this paper, we describe several techniques for detecting tonic pitch value in Indian classical music. In Indian music, the raga is the basic melodic framework and it is built on the tonic. Tonic detection is therefore fundamental for any melodic analysis in Indian classical music. This work/nexplores detection of tonic by processing the pitch histograms of Indian classic music. Processing of pitch histograms using group delay functions and its ability to amplify certain traits of Indian music in the pitch histogram, is discussed. Three different strategies to detect tonic, namely, the concert method, the template matching and segmented histogram method are proposed. The concert method exploits the fact that the tonic is constant over a piece/concert./ntemplatematchingmethod and segmented histogrammethods/nuse the properties: (i) the tonic is always present in the background, (ii) some notes are less inflected and dominant, to detect the tonic of individual pieces. All the three methods yield good results for Carnatic music (90−100% accuracy), while for Hindustanimusic, the templatemethod works best, provided the v¯adi samv¯adi notes for a given piece are known (85%)., This research was partly funded by the European Research Council/nunder the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement/n267583)
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Opportunities for a cultural specific approach in the computational description of music

  • Serra, Xavier
Comunicació presentada al 2nd CompMusic Workshop, celebrat els dies 12 i 13 de juliol de 2012 a Istanbul (Turquia), organitzat per CompMusic., The current research in Music Information Retrieval (MIR) is showing the potential that the Information Technologies can have in music related applications. A/nmajor research challenge in that direction is how to automatically/ndescribe/annotate audio recordings and how to use the resulting descriptions to discover and appreciate music in new ways. But music is a complex phenomenon/nand the description of an audio recording has to deal with this complexity. For example, each music/nculture has specificities and emphasizes different musical/nand communication aspects, thus the musical recordings of each culture should be described differently. At the same time these cultural specificities give us the opportunity to pay attention to musical concepts and/nfacets that, despite being present in most world musics, are not easily noticed by listeners. In this paper we present some of the work done in the CompMusic project, including ideas and specific examples on how to take advantage of the cultural specificities of different/nmusical repertoires. We will use examples from the art music traditions of India, Turkey and China., The CompMusic project is funded by the European Research Council under the European Union’s Seventh/nFramework Programme (FP7/2007-2013) / ERC grant agreement 267583.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Automatic lyrics-to-audio alignment in classical Turkish music

  • Dzhambazov, Georgi Bogomilov
  • Sentürk, Sertan
  • Serra, Xavier
Comunicació presentada al Fourth International Workshop on Folk Music Analysis (FMA2014), celebrat els dies 12 i 13 de juny de 2014 a Istanbul, Turquia., We apply a lyrics-to-audio alignment state-of-the-art approach to polyphonic pieces from classical Turkish repertoire. A phonetic recognizer is employed, whereby each phoneme is assigned a hidden Markov model (HMM). Initially trained on speech, the models are adapted on singing voice to match the acoustic characteristics of the test dataset. Being the first study on lyrics-to-audio alignment applied on Turkish music, it could serve as a baseline for singing material with similar musical characteristics. As part of this work a dataset of recordings from the classical music tradition is compiled. Experiments, conducted separately for male and female singers, show that female singing is aligned more accurately., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Discovery of syllabic percussion patterns in tabla solo recordings

  • Gupta, Swapnil
  • Srinivasamurthy, Ajay
  • Kumar, Manoj
  • Murthy, Hema A.
  • Serra, Xavier
We address the unexplored problem of percussion pattern/ndiscovery in Indian art music. Percussion in Indian art music uses onomatopoeic oral mnemonic syllables for the transmission of repertoire and technique. This is utilized for the/ntask of percussion pattern discovery from audio recordings./nFrom a parallel corpus of audio and expert curated scores/nfor 38 tabla solo recordings, we use the scores to build a/nset of most frequent syllabic patterns of different lengths./nFrom this set, we manually select a subset of musically representative query patterns. To discover these query patterns/nin an audio recording, we use syllable-level hidden Markov/nmodels (HMM) to automatically transcribe the recording/ninto a syllable sequence, in which we search for the query/npattern instances using a Rough Longest Common Subsequence (RLCS) approach. We show that the use of RLCS/nmakes the approach robust to errors in automatic transcrip-/ntion, significantly improving the pattern recall rate and F-/nmeasure. We further propose possible enhancements to improve the results., This work is partly supported by the European Research/nCouncil under the European Union’s Seventh Framework/nProgram, as a part of the CompMusic project (ERC grant/nagreement 267583)
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Characterization of embellishments in ney performances of makam music in Turkey

  • Özaslan, Tan Hakan
  • Serra, Xavier
  • Arcos, Josep Lluís
The embellishments of makam music in Turkey are an/ninherent characteristic of the music rather than a separate/nexpressive resource, thus their understanding is essential to/ncharacterize this music. We do a computational study, in/nwhich we analyze audio recordings of 8 widely acknowledged/nTurkish ney players covering the period from the/nyear 1920 to 2000. From the extracted fundamental frequency,/nwe manually segment and identify 327 separate/nembellishments of the types vibrato and kaydırma. We analyze/nthem and characterize the behavior of two features/nthat help us differentiate performance styles, namely vibrato/nrate change and pitch bump. Also we compare these/nembellishments with the ones used in Western classical/nmusic. With our approach, we have an explicit and formalized/nway to understand ney embellishments, which is a/nstep towards the automatic characterization of makam music/nin Turkey.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

NACTA: construyendo el futuro de la tradición del jingju

  • Caro Repetto, Rafael
  • Serra, Xavier
Comunicació presentada al XIV Congreso de la Sociedad de Etnomusicología, IX congreso de IASPM celebrat del 16 al 19 de novembre de 2016 a Madrid, Espanya., En sus poco más de doscientos años de historia, el jingju, u ópera de Pekín, ha conseguido el título de tesoro nacional (guocui) y como tal ha asumido la turbulenta historia reciente del país. En la puja por situarse como potencia internacional, China se esfuerza en promover este emblema cultural ante una audiencia global, consiguiendo de hecho haber sido incluido en la Lista Representativa del Patrimonio Cultural Inmaterial de la Humanidad, al tiempo que la audiencia local es cada vez mayor en edad, pero menor en número. En el centro de estas tensiones sociales y políticas, y en cierta medida ajena a las mismas, se encuentra la National Academy of Chinese Theatre Arts (Zhongguo Xiqu Xueyuan, NACTA), la principal institución de educación superior para actores, actrices, músicos, directores, compositores, escenógrafos y todo profesional relacionado con el jingju. Su cometido es el de preservar la tradición que ha de identificar el legado cultural con el que el país se presenta ante la comunidad global, la misma que se aleja cada vez más de su sociedad contemporánea. Como resultado del trabajo de campo desarrollado en este centro, esta ponencia estudia NACTA como espacio de negociación entre las diferentes dinámicas que operan sobre este género. A partir de entrevistas y conversaciones con alumnos y profesores, y de la participación directa en sus actividades formativas, los autores analizan los cambios en los modelos de formación y su repercusión en la transmisión / recreación de la tradición, así como la influencia de las políticas culturales, los continuos cambios sociales, la competencia con nuevos modelos de consumo de ocio, la (falta de) incorporación de las nuevas tecnologías y la idiosincrasia de las nuevas generaciones de jóvenes intérpretes frente a estos retos.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Corpora for music information research in Indian art music

  • Srinivasamurthy, Ajay
  • Koduri, Gopala Krishna
  • Gulati, Sankalp
  • Ishwar, Vignesh
  • Serra, Xavier
Comunicació presentada a la International Computer Music Conference, ICMC/SMC, celebrada a Atenes (Grècia) del 14 al 20 de setembre de 2014., Research corpora are representative collections of data and are essential to develop data-driven approaches in Music Information Research (MIR). We address the problem of building research corpora for MIR in Indian art music traditions of Hindustani and Carnatic music, considering several relevant criteria for building such corpora. We also discuss a methodology to assess the corpora based on these criteria and present an evaluation of the corpora in their coverage and completeness. In addition to the corpora, we briefly describe the test datasets that we have built for use in many research tasks. In specific, we describe the tonic dataset, the Carnatic rhythm dataset, the Carnatic varṇaṁ dataset, and the Mridangam stroke dataset. The criteria and the evaluation methodology discussed in this article can be used to systematically build a representative and comprehensive research corpus. The corpora and the datasets are accessible to the research community from a central online repository., The compilation of the corpora and the datasets presented in this paper has been a collective effort of the members of the CompMusic project and a number of collaborators. The CompMusic project is funded by the European Research Council under the European Union’s Seventh Framework Program (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Score-informed syllable segmentation for a cappella singing voice with convolutional neural networks

  • Pons Puig, Jordi
  • Gong, Rong
  • Serra, Xavier
Comunicació presentada a la 18th International Society for Music Information Retrieval Conference (ISMIR 2017), celebrada els dies 23 a 27 d'octubre de 2017 a Suzhou, Xina., This paper introduces a new score-informed method for the segmentation of jingju a cappella singing phrase into syllables. The proposed method estimates the most likely sequence of syllable boundaries given the estimated syllable onset detection function (ODF) and its score. Throughout the paper, we first examine the jingju syllables structure and propose a definition of the term “syllable onset”. Then, we identify which are the challenges that jingju a cappella singing poses. Further, we investigate how to improve the syllable ODF estimation with convolutional neural networks (CNNs). We propose a novel CNN architecture that allows to efficiently capture different timefrequency scales for estimating syllable onsets. Besides, we propose using a score-informed Viterbi algorithm – instead of thresholding the onset function–, because the available musical knowledge we have (the score) can be used to inform the Viterbi algorithm to overcome the identified challenges. The proposed method outperforms the state-of-the-art in syllable segmentation for jingju a cappella singing. We further provide an analysis of the segmentation errors which points possible research directions., This work is partially supported by the Maria de Maeztu Programme (MDM-2015-0502) and the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Folksonomy-based tag recommendation for online audio clip sharing

  • Font Corbera, Frederic
  • Serrà Julià, Joan
  • Serra, Xavier
Collaborative tagging has emerged as an efficient way to/nsemantically describe online resources shared by a community/nof users. However, tag descriptions present some/ndrawbacks such as tag scarcity or concept inconsistencies./nIn these situations, tag recommendation strategies can help/nusers in adding meaningful tags to the resources being described./nFreesound is an online audio clip sharing site that/nuses collaborative tagging to describe a collection of more/nthan 140,000 sound samples. In this paper we propose four/nalgorithm variants for tag recommendation based on tag/nco-occurrence in the Freesound folksonomy. On the basis/nof removing a number of tags that have to be later predicted/nby the algorithms, we find that using ranks instead of raw/ntag similarities produces statistically significant improvements./nMoreover, we show how specific strategies for selecting/nthe appropriate number of tags to be recommended/ncan significantly improve algorithms’ performance. These/ntwo aspects provide insight into some of the most basic/ncomponents of tag recommendation systems, and we plan/nto exploit them in future real-world deployments., This work is partially supported under BES-2010-037309/nFPI grant from the Spanish Ministry of Science and Innovation/nfor the TIN2009- 14247-C02-01 DRIMS project. JS/nacknowledges 2009-SGR-1434 from Generalitat de Catalunya,/nJAEDOC069/2010 from Consejo Superior de Investigaciones/nCient´ıficas, TIN2009-13692-C03-01 from the/nSpanish Government, and EU Feder Funds.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Quantitative analysis of the relationship between linguistic tones and melody in jingju using music scores

  • Caro Repetto, Rafael
  • Zhang, Shuo
  • Serra, Xavier
Comunicació presentada a: 4th International Digital Libraries for Musicology workshop (DLfM 2017), celebrat el 28 d'octubre de 2017 a Shanghai, Xina., When lyrics of tonal languages are set to music, the pitch contour of the tones has to agree to a certain extent with the melodic contour to assure intelligibility. The relationship between the linguistic tones of the complex dialectal construct used in jingju (commonly known as Beijing or Peking opera) and its melody has been largely studied, but not definite consensus has been achieved among scholars. After reviewing the related literature, we present a first approach for the quantitative analysis of the relationship between linguistic tones and melody in jingju using a collection of machine readable music scores with tone category annotations for 7,283 syllables. We describe two statistical analyses performed in this collection regarding the melodic contour for each syllable and the pitch height relationship in 5,494 pairs of consecutive syllables. We argue that the obtained results contribute to supporting claims from the literature and complementing others, although some limitations of the approach might nuance the confidence of their validity., This research is funded by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Audio to score matching by combining phonetic and duration information

  • Gong, Rong
  • Pons Puig, Jordi
  • Serra, Xavier
Comunicació presentada a la 18th International Society for Music Information Retrieval Conference (ISMIR 2017), celebrada els dies 23 a 27 d'octubre de 2017 a Suzhou, Xina., We approach the singing phrase audio to score matching problem by using phonetic and duration information – with a focus on studying the jingju a cappella singing case. We argue that, due to the existence of a basic melodic contour for each mode in jingju music, only using melodic information (such as pitch contour) will result in an ambiguous matching. This leads us to propose a matching approach based on the use of phonetic and duration information. Phonetic information is extracted with an acoustic model shaped with our data, and duration information is considered with the Hidden Markov Models (HMMs) variants we investigate. We build a model for each lyric path in our scores and we achieve the matching by ranking the posterior probabilities of the decoded most likely state sequences. Three acoustic models are investigated: (i) convolutional neural networks (CNNs), (ii) deep neural networks (DNNs) and (iii) Gaussian mixture models (GMMs). Also, two duration models are compared: (i) hidden semi-Markov model (HSMM) and (ii) post-processor duration model. Results show that CNNs perform better in our (small) audio dataset and also that HSMM outperforms the post-processor duration model., This work is partially supported by the Maria de Maeztu Programme (MDM-2015-0502) and by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Creating research corpora for the computational study of music: the case of the CompMusic Project

  • Serra, Xavier
Comunicació presentada a la 53rd International Conference: Semantic audio, celebrada els dies 27 a 29 de gener de 2014 a Londres, Regne Unit., A fundamental concern in music information research is the use of appropriate data sets, research corpora, from which to perform the needed data processing tasks. These corpora have to be suited for the specific research problems to be addressed and the design criteria with which to create them is a research task to which not much attention has been paid. In the CompMusic project we are studying several non-western art music traditions and a major effort has been the creation of appropriate data collections with which to study and characterise the melodic and rhythmic aspects of these traditions. In this article we go over the criteria used to create these collections and we describe the specificities of each of the collections gathered., The CompMusic project is funded by the European Research Council under the European Unions Seventh Framework Program (FP7/2007-2013) / ERC grant agreement 267583.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Culture-aware approaches to modeling and description of intonation using multimodal data

  • Koduri, Gopala Krishna
Comunicació presentada a la International Conference on Knowledge Engineering and Knowledge Management (EKAW), que es va celebrar del 24 al 28 de novembre de 2014 a Linkoping, Suècia., Computational approaches that conform to the cultural context are of paramount importance in music information research. The current state-of-the-art has a limited view of such context, which manifests in our ontologies, data-, cognition- and interaction-models that are biased to the market-driven popular music. In a step towards addressing this, the thesis draws upon multimodal data sources concerning art music traditions, extracting culturally relevant and musically meaningful information about melodic intervals from each of them and structuring it with formal knowledge representations. As part of this, we propose novel approaches to describe intonation in audio music recordings and to use and adapt the semantic web infrastructure to complement this with the knowledge extracted from text data. Due to the complementary nature of the data sources, structuring and linking the extracted information results in a symbiosis mutually enriching their information. Over this multimodal knowledge base, we propose similarity measures for the discovery of musical entities, yielding a culturally-sound navigation space., This thesis is supervised by Dr. Xavier Serra, and was partly funded by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Identification of potential Music Information Retrieval technologies for computer-aided jingju singing training

  • Gong, Rong
  • Serra, Xavier
Comunicació presentada a: 5th China Conference on Sound and Music Technology - Chinese Traditional Music Technology Session celebrada el 21 de novembre de 2017 a Suzhou, Xina., Music Information Retrieval (MIR) technologies have been proven useful in assisting western classical singing training. Jingju (also known as Beijing or Peking opera) singing is different from western singing in terms of most of the perceptual dimensions, and the trainees are taught by using mouth/heart method. In this paper, we first present the training method used in the professional jingju training classroom scenario and show the potential benefits of introducing the MIR technologies into the training process. The main part of this paper dedicates to identify the potential MIR technologies for jingju singing training. To this intent, we answer the question: how the jingju singing tutors and trainees value the importance of each jingju musical dimension—intonation, rhythm, loudness, tone quality and pronunciation? This is done by (i) classifying the classroom singing practices, tutor's verbal feedbacks into these 5 dimensions, (ii) surveying the trainees. Then, with the help of the music signal analysis, a finer inspection on the classroom practice recording examples reveals the detailed elements in the training process. Finally, based on the above analysis, several potential MIR technologies are identified and would be useful for the jingju singing training., This by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Modeling of phoneme durations for alignment between polyphonic audio and lyrics

  • Dzhambazov, Georgi Bogomilov
  • Serra, Xavier
Comunicació presentada al 12th Sound and Music Computing Conference, celebrada del 30 de juliol a l'1 d'agost 2015 a Maynooth (Irlanda)., In this work we propose how to modify a standard scheme for text-to-speech alignment for the alignment of lyrics and singing voice. To this end we model the duration of phonemes specific for the case of singing. We rely on a duration-explicit hidden Markov model (DHMM) phonetic recognizer based on mel frequency cepstral coefficients (MFCCs), which are extracted in a way robust to background instrumental sounds. The proposed approach is tested on polyphonic audio from the classical Turkish music tradition in two settings: with and without modeling phoneme durations. Phoneme durations are inferred from sheet music. In order to assess the impact of the polyphonic setting, alignment is evaluated as well on an acapella dataset, compiled especially for this study. We show that the explicit modeling of phoneme durations improves alignment accuracy by absolute 10 percent on the level of lyrics lines (phrases) and performs on par with state-of-the-art aligners for other languages., This work is partly supported by the European Research/nCouncil under the European Unions Seventh Framework/nProgram, as part of the CompMusic project (ERC grant/nagreement 267583) and partly by the AGAUR research grant.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Composition identification in Ottoman-Turkish makam music using transposition-invariant partial audio-score alignment

  • Sentürk, Sertan
  • Serra, Xavier
The composition information of audio recordings is highly valuable for many tasks such as automatic music description and music discovery. Given a music collection, two typical scenarios are retrieving the composition(s) performed in an audio recording and retrieving the audio recording( s), where a composition is performed. We present a composition identification methodology for these two tasks, which makes use of music scores. Our methodology first attempts to align a fragment of the music score of a composition with an audio recording. Next, it computes a similarity from the best obtained alignment. True audio-score pair emits a high similarity value. We repeat this procedure between all audio recordings and music scores, and filter the true pairs by a simple approach using logistic regression. The methodology is specialized according to the cultural-specific aspects of Ottoman-Turkish makam music (OTMM), achieving 0.96 and 0.95 mean average precision (MAP) for composition retrieval and performance retrieval tasks, respectively. We hope that our method would be useful in creating semantically linked music corpora for cultural heritage and preservation, semantic web applications and musicological studies., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A generalized Bayesian model for tracking long metrical cycles in acoustic music signals

  • Srinivasamurthy, Ajay
  • Holzapfel, Andre
  • Cemgil, Ali Taylan
  • Serra, Xavier
Most musical phenomena involve repetitive structures that enable listeners to track meter, i.e. the tactus or beat, the longer over-arching measure or bar, and possibly other related layers. Meters with long measure duration, sometimes lasting more than a minute, occur in many music cultures, e.g. from India, Turkey, and Korea. However, current meter tracking algorithms, which were devised for cycles of a few seconds length, cannot process such structures accurately. We present a novel generalization to an existing Bayesian model for meter tracking that overcomes this limitation. The proposed model is evaluated on a set of Indian Hindustani music recordings, and we document significant performance increase over the previous models. The presented model opens the way for computational analysis of performances with long metrical cycles, and has important applications in music studies as well as in commercial applications that involve such musics., This work is partly supported by the European Research Council as part of the CompMusic project (ERC grant agreement 267583), and Vienna Science and Technology Fund (WWTF, project MA14-018).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Rāga Recognition based on Pitch Distribution Methods

  • Koduri, Gopala Krishna
  • Gulati, Sankalp
  • Rao, Preeti
  • Serra, Xavier
  • Serra, Xavier
Rāga forms the melodic framework for most of the music of the Indian subcontinent. Thus automatic rāga recognition is a fundamental step in the computational modeling of the Indian art-music traditions. In this work, we investigate the properties of rāga and the natural processes by which people identify it. We bring together and discuss the previous computational approaches to rāga recognition correlating them with human techniques, in both Karṇāṭak (south Indian) and Hindustānī (north Indian) music traditions. Thee approaches which are based on first-order pitch distributions are further evaluated on a large comprehensive dataset to understand their merits and limitations. We outline the possible short and mid-term future directions in this line of work., This research was partly funded by the European Research Council under the European Union's Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Creating corpora for computational research in arab-andalusian music

  • Sordo, Mohamed
  • Chaachoo, Amin
  • Serra, Xavier
Comunicació presentada al 1st International workshop on Digial Libraries for Musicology, que va tenir lloc el 12 de setembre de 2014 a Longres, Regne Unit., Research corpora are fundamental for the computational study of music. The design criteria with which to create them is a research task in itself. These corpora need to be well suited for the specific research problems to be addressed. Since these research problems are also shaped by musical, cultural and other specific aspects of the music traditions to be studied, the research corpora should take these specificities into account. In this paper we address the problems of creating corpora for computational research on Arab-Andalusian music, considering several relevant criteria for creating such corpora. We focus on the problems raised during the annotation process of the corpora, specifically the language issues surrounding this art music tradition. Following the criteria, we created a research corpus consisting of audio recordings with their corresponding metadata, lyrics and music scores. So far we have gathered 338 recordings from 3 different Arab-Andalusian music schools of Morocco, covering most of the musical modes, rhythms and forms of this art music tradition. The Arab-Andalusian corpus is accessible to the research community from a central online repository. Moreover, the audio recordings of this corpora are freely available through the Internet Archive repository. The Arab- Andalusian corpus can be used to generate test datasets, which can be used as ground truth to test several computational research tasks., The compilation of the corpora presented in this paper has been a collective effort of the members of the CompMusic project. The CompMusic project is funded by the European Research Council under the European Union’s Seventh Framework Program (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A corpus for computational research of Turkish Makam music

  • Uyar, Burak
  • Atli, Hasan Sercan
  • Sentürk, Sertan
  • Bozkurt, Baris
  • Serra, Xavier
Comunicació presentada al 1st International workshop on Digial Libraries for Musicology, que va tenir lloc el 12 de setembre de 2014 a Longres, Regne Unit., Each music tradition has its own characteristics in terms of melodic, rhythmic and timbral properties as well as semantic understandings. To analyse, discover and explore these culture-specific characteristics, we need music collections which are representative of the studied aspects of the music tradition. For Turkish makam music, there are various resources available such as audio recordings, music scores, lyrics and editorial metadata. However, most of these resources are not typically suited for computational analysis, are hard to access, do not have sufficient quality or do not include adequate descriptive information. In this paper we present a corpus of Turkish makam music created within the scope of the CompMusic project. The corpus is intended for computational research and the primary considerations during the creation of the corpus reflect some criteria, namely, purpose, coverage, completeness, quality and re-usability. So far, we have gathered approximately 6000 audio recordings, 2200 music scores with lyrics and 27000 instances of editorial metadata related to Turkish makam music. The metadata include information about makams, recordings, scores, compositions, artists etc. as well as the interrelations between them. In this paper, we also present several test datasets of Turkish makam music. Test datasets contain manual annotations by experts and they provide ground truth for specific computational tasks to test, calibrate and improve the research tools. We hope that this research corpus and the test datasets will facilitate academic studies in several fields such as music information retrieval and computational musicology., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A multipitch approach to tonic identification in Indian classical music

  • Salamon, Justin
  • Gulati, Sankalp
  • Serra, Xavier
The tonic is a fundamental concept in Indian classical music/nsince it constitutes the base pitch from which a lead/nperformer constructs the melodies, and accompanying instruments/nuse it for tuning. This makes tonic identification/nan essential first step for most automatic analyses of Indian/nclassical music, such as intonation and melodic analysis,/nand raga recognition. In this paper we address the task/nof automatic tonic identification. Unlike approaches that/nidentify the tonic from a single predominant pitch track,/nhere we propose a method based on a multipitch analysis of/nthe audio. We use a multipitch representation to construct/na pitch histogram of the audio excerpt, out of which the/ntonic is identified. Rather than manually define a template,/nwe employ a classification approach to automatically learn/na set of rules for selecting the tonic. The proposed method/nreturns not only the pitch class of the tonic but also the precise/noctave in which it is played. We evaluate the approach/non a large collection of Carnatic and Hindustani music, obtaining/nan identification accuracy of 93%. We also discuss/nthe types of errors made by our proposed method, as well/nas the challenges in generating ground truth annotations., This research/nwas funded by the Programa de Formación del Profesorado/nUniversitario (FPU) of the Ministerio de Educación de España and the European Research Council under/nthe European Union’s Seventh Framework Programme/n(FP7/2007-2013) / ERC grant agreement 267583 (Comp-/nMusic).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Towards alignment of score and audio recordings of ottoman-turkish makam music

  • Sentürk, Sertan
  • Gulati, Sankalp
  • Serra, Xavier
Comunicació presentada al Fourth International Workshop on Folk Music Analysis (FMA2014), celebrat els dies 12 i 13 de juny de 2014 a Istanbul, Turquia., Audio-score alignment is a multi-modal task, which facilitates many related tasks such as intonation analysis, structure analysis and automatic accompaniment. In this paper, we present a audio-score alignment methodology for the classical Ottoman-Turkish music tradition. Given a music score of a composition with structure (section) information and an audio performance of the same composition, our method first extracts a synthetic prominent pitch per section from the note values and durations in the score and a audio prominent pitch from the audio recording. Then it identifies the performed tonic frequency by using melodic information in the repetitive section in the score. Next it links each section with the time intervals where each section performed in the audio recording (i.e. structure level alignment) by comparing the extracted pitch features. Finally the score and the audio recordings are aligned in the note-level. For the initial experiments we chose DTW, a standard technique used in audio-score alignment, to show how well the state-of-the-art performs in makam musics. The results show that our method is able to handle the tonic transpositions and the structural differences with ease, however improvements, which address the characteristics of the music scores and the performances of makam musics, are needed in our note-level alignment methodology. To the best knowledge this paper presents the first audio-score alignment method proposed for makam musics., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

An Analysis and storage system for music research datasets

  • Porter, Alastair
  • Serra, Xavier
Comunicació presentada al 1st International workshop on Digial Libraries for Musicology, que va tenir lloc el 12 de setembre de 2014 a Longres, Regne Unit., We present a work ow processing and data storage system that has been developed to store the computational analysis of large databases of music-related documents. Documents can consist of any music-related data, for example, audio les or symbolic scores. The system can be used to run fea- ture extraction algorithms to compute new data based on these input documents. The feature extraction process can be distributed over many networked computers to reduce calculation time. Researchers can develop and run their own algorithms on a collection of music-related documents using any programming system without needing system adminis- trator knowledge or direct access to the les, using a web based interface. Computed features are stored in the sys- tem along with a reference to the version of the code that was used to compute it. Audio les and computed features can be accessed over the internet. This system has been developed as a supporting part of the CompMusic project, allowing collaborating researchers to develop new feature ex- traction algorithms and share them with other researchers. It has been released under an open-source licence., The CompMusic project is funded by the European Research Council under the European Union's Seventh Framework Program (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Discovering rāga motifs by characterizing communities in networks of melodic patterns

  • Gulati, Sankalp
  • Serrà Julià, Joan
  • Ishwar, Vignesh
  • Serra, Xavier
Comunicació presentada a la 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), celebrada els dies 20 a 25 de març a Xangai, Xina., Ra̅ga motifs are the main building blocks of the melodic structures in Indian art music. Therefore, the discovery and characterization of such motifs is fundamental for the computational analysis of this music. We propose an approach for discovering ra̅ga motifs from audio music collections. First, we extract melodic patterns from a collection of 44 hours of audio comprising 160 recordings belonging to 10 ra̅gas. Next, we characterize these patterns by performing a network analysis, detecting non-overlapping communities, and exploiting the topological properties of the network to determine a similarity threshold. With that, we select a number of motif candidates that are representative of a ra̅ga, the ra̅ga motifs. For a formal evaluation we perform listening tests with 10 professional musicians. The results indicate that, on an average, the selected melodic phrases correspond to ra̅ga motifs with 85% positive ratings. This opens up the possibilities for many musically-meaningful computational tasks in Indian art music, including human-interpretable ra̅ga recognition, semantic-based music discovery, or pedagogical tools., This work is partly supported by the European Research Council under the European Unions Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Comparision of the singing style of two jingju schools

  • Caro Repetto, Rafael
  • Gong, Rong
  • Kroher, Nadine
  • Serra, Xavier
Comunicació presentada a la 16th International Society for Music Information Retrieval Conference (ISMIR 2015), celebrada els dies 26 a 30 d'octubre de 2015 a Màlaga, Espanya., Performing schools (liupai) in jingju (also known as Peking or Beijing opera) are one of the most important elements for the appreciation of this genre among connoisseurs. In the current paper, we study the potential of MIR techniques for supporting and enhancing musicological descriptions of the singing style of two of the most renowned jingju schools for the dan role-type, namely Mei and Cheng schools. To this aim, from the characteristics commonly used for describing singing style in musicological literature, we have selected those that can be studied using standard audio features. We have selected eight recordings from our jingju music research corpus and have applied current algorithms for the measurement of the selected features. Obtained results support the descriptions from musicological sources in all cases but one, and also add precision to them by providing specific measurements. Besides, our methodology suggests some characteristics not accounted for in our musicological sources. Finally, we discuss the need for engaging jingju experts in our future research and applying this approach for musicological and educational purposes as a way of better validating our methodology., This research is funded by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Features for analysis of Makam music

  • Bozkurt, Baris
For computational studies of makam music, it is essential to gather a list of characteristics that constitute a makam and explore corresponding quantitative features for automatic/nanalysis. This study is such an attempt where we address the characteristics for makams as defined in theory books and deduce a list of quantitative features. The target here is to evoke discussions on some measurable features other than providing complete analysis on the/ndiscriminative potentials of each proposed feature which could be the subject of a few larger studies., This work was funded by the European Research Council/nunder the European Union’s Seventh Framework Programme/n(FP7/2007-2013) / ERC grant agreement 267583/n(CompMusic)
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

The computational study of a musical culture through its digital traces

  • Serra, Xavier
From most musical cultures there are digital traces, digital artifacts, that can be processed and studied computationally, and this has been the focus of computational musicology already for several decades. This type of research requires clear formalizations and some simplifications, for example, by considering that a musical culture can be conceptualized as a system of interconnected entities. A musician, an instrument, a performance, or a melodic motive, are examples of entities and they are linked by various types of relationships. We then need adequate digital traces of the entities, for example, a textual description can be a useful trace of a musician and a recording one of a performance. The analytical study of these entities and of their interactions is accomplished by processing the digital traces and by generating mathematical representations and models of them. But a more ambitious goal, however, is to go beyond the study of individual artifacts and analyze the overall system of interconnected entities in order to model a musical culture as a whole. The reader might think that this is science fiction, and he or she might be right, but there is research trying to make advances in this direction. In this article I undertake an overview the state-of-the-art related to this type of research, identifying current challenges, describing computational methodologies being developed, and summarizing musicologically relevant results of such research. In particular, I review the work done within CompMusic, a project in which my colleagues and I have developed audio signal processing, machine learning, and semantic web methodologies to study several musical cultures., The CompMusic project has been funded by the European Research Council under the European Union’s Seventh Framework Program (FP7/2007-2013) / ERC grant agreement 267583.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Particle filters for efficient meter tracking with dynamic bayesian networks

  • Srinivasamurthy, Ajay
  • Holzapfel, Andre
  • Cemgil, Ali Taylan
  • Serra, Xavier
Comunicació presentada a la 16th International Society for Music Information Retrieval Conference (ISMIR 2015), celebrada els dies 26 a 30 d'octubre de 2015 a Màlaga, Espanya., Recent approaches in meter tracking have successfully applied Bayesian models. While the proposed models can be adapted to different musical styles, the applicability of these flexible methods so far is limited because the application of exact inference is computationally demanding. More efficient approximate inference algorithms using particle filters (PF) can be developed to overcome this limitation. In this paper, we assume that the type of meter of a piece is known, and use this knowledge to simplify an existing Bayesian model with the goal of incorporating a more diverse observation model. We then propose Particle Filter based inference schemes for both the original model and the simplification. We compare the results obtained from exact and approximate inference in terms of meter tracking accuracy as well as in terms of computational demands. Evaluations are performed using corpora of Carnatic music from India and a collection of Ballroom dances. We document that the approximate methods perform similar to exact inference, at a lower computational cost. Furthermore, we show that the inference schemes remain accurate for long and full length recordings in Carnatic music., This work is supported by the European Research Council (grant number 267583) and a Marie Curie Intra-European Fellowship (grant number 328379).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Automatic tonic identification in Indian art music: approaches and evaluation

  • Gulati, Sankalp
  • Bellur, Ashwin
  • Salamon, Justin
  • Ranjani, H. G.
  • Ishwar, Vignesh
  • Murthy, Hema A.
  • Serra, Xavier
The tonic is a fundamental concept in Indian art music. It is the base pitch, which an artist chooses in order to construct the melodies during a rāg(a) rendition, and all accompanying instruments are tuned using the tonic pitch. Consequently, tonic identification is a fundamental task for most computational analyses of Indian art music, such as intonation analysis, melodic motif analysis and rāg recognition. In this paper we review existing approaches for tonic identification in Indian art music and evaluate them on six diverse datasets for a thorough comparison and analysis. We study the performance of each method in different contexts such as the presence/absence of additional metadata, the quality of audio data, the duration of audio data, music tradition (Hindustani/Carnatic) and the gender of the singer (male/female). We show that the approaches that combine multi-pitch analysis with machine learning provide the best performance in most cases (90% identification accuracy on average), and are robust across the aforementioned contexts compared to the approaches based on expert knowledge. In addition, we also show that the performance of the latter can be improved when additional metadata is available to further constrain the problem. Finally, we present a detailed error analysis of each method, providing further insights into the advantages and limitations of the methods., This work is partly supported by the European Research Council/nunder the European Union’s Seventh Framework Program, as/npart of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Metrical-accent aware vocal onset detection in polyphonic audio

  • Dzhambazov, Georgi Bogomilov
  • Holzapfel, Andre
  • Srinivasamurthy, Ajay
  • Serra, Xavier
Comunicació presentada a la ISMIR 2017: 18th International Society for Music Information Retrieval Conference, celebrada els dies 23 a 27 d'octubre de 2017 a Suzhou, Xina., The goal of this study is the automatic detection of onsets of the singing voice in polyphonic audio recordings. Starting with a hypothesis that the knowledge of the current position in a metrical cycle (i.e. metrical accent) can improve the accuracy of vocal note onset detection, we propose a novel probabilistic model to jointly track beats and vocal note onsets. The proposed model extends a state of the art model for beat and meter tracking, in which a-priori probability of a note at a specific metrical accent interacts with the probability of observing a vocal note onset. We carry out an evaluation on a varied collection of multi-instrument datasets from two music traditions (English popular music and Turkish makam) with different types of metrical cycles and singing styles. Results confirm that the proposed model reasonably improves vocal note onset detection accuracy compared to a baseline model that does not take metrical position into account., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583) and partly by the Spanish Ministry of Economy and Competitiveness, through the ”Mar´ıa de Maeztu” Programme for Centres/Units of Excellence in R&D” (MDM-2015-0502).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

What is the effect of audio quality on the robustness of MFCCs and chroma features?

  • Urbano, Julián
  • Bogdanov, Dmitry
  • Herrera Boyer, Perfecto, 1964-
  • Gómez Gutiérrez, Emilia, 1975-
  • Serra, Xavier
Music Information Retrieval is largely based on descriptors computed from audio signals, and in many practical applications they are to be computed on music corpora containing audio files encoded in a variety of lossy formats. Such encodings distort the original signal and therefore may affect the computation of descriptors. This raises the question of the robustness of these descriptors across various audio encodings. We examine this assumption for the case of MFCCs and chroma features. In particular, we analyze their robustness to sampling rate, codec, bitrate, frame size and music genre. Using two different audio analysis tools over a diverse collection of music tracks, we compute several statistics to quantify the robustness of the resulting descriptors, and then estimate the practical effects for a sample task like genre classification., This work is partially supported by an A4U postdoctoral grant and projects SIGMUS (TIN2012-36650), Comp- Music (ERC 267583), PHENICX (ICT-2011.8.2) and GiantSteps (ICT-2013-10).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Detection of Raga-characteristic phrases from Hindustani Classical Music Audio

  • Ross, Joe Cheri
  • Rao, Preeti
Melodic motifs form essential building blocks in Indian Classical music. The motifs, or key phrases, provide/nstrong cues to the identity of the underlying raga in both Hindustani and Carnatic styles of Indian music. Automatic identification and clustering of similar motifs is relevant in this context. The inherent variations in various instances of a characteristic phrase in a bandish (composition)/nperformance make it challenging to identify similar phrases in a performance. A nyas svara (long note)/nmarks the ending of these phrases. The proposed method does segmentation of phrases through identification of/nnyas and computes similarity with the reference characteristic phrase.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A score-informed computational description of svaras using a statistical model

  • Sentürk, Sertan
  • Koduri, Gopala Krishna
  • Serra, Xavier
Comunicació presentada a: 13th Sound and Music Computing Conference (SMC 2016), celebrat a Hamburg (Alemanya), del 31 d'agost a 3 de setembre de 2016., Musical notes are often modeled as a discrete sequence of points on a frequency spectrum with possibly different interval sizes such as just-intonation. Computational descriptions abstracting the pitch content in audio music recordings have used this model, with reasonable success in several information retrieval tasks. In this paper, we argue that this model restricts a deeper understanding of the pitch content. First, we discuss a statistical model of musical notes which widens the scope of the current one and opens up possibilities to create new ways to describe the pitch content. Then we present a computational approach that partially aligns the audio recording with its music score in a hierarchical manner first at metrical cyclelevel and then at note-level, to describe the pitch content using this model. It is evaluated extrinsically in a classification test using a public dataset and the result is shown to be significantly better compared to a state-of-the-art approach. Further, similar results obtained on a more challenging dataset which we have put together, reinforces that our approach outperforms the other., This research was partly funded by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Carnatic music: Svara, Gamaka, Motif and Raga identity

  • Krishna, T. M.
  • Ishwar, Vignesh
Comunicació presentada al 2nd CompMusic Workshop, celebrat els dies 12 i 13 de juliol de 2012 a Istanbul (Turquia), organitzat per CompMusic., Over the last century in Karnatik music, the method of/nunderstanding raga has been to break it down into its various/ncomponents, svara, scale, gamaka, and phrases. In this paper, an attempt is made to define the abstract concept of raga in its entirety within the aesthetics of Karnatik music considering the various components and their symbiotic relationship. This paper also attempts to prove that the/nidentity of a raga exists as a whole. Section 2 explains the/nconcept of a fundamental musical note or svara. Section 3 illustrates the concept of gamaka or inflections. Section 4 delves into the concept of raga in detail and then flows into Section 5 which enunciates the identity of a raga in terms of svara, gamaka, and phraseology. The paper concludes in Section 6, and Section 7 gives the references.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Automatic creation of knowledge graphs from digital musical document libraries

  • Oramas, Sergio
  • Sordo, Mohamed
  • Serra, Xavier
Comunicació presentada a la 9th Conference on Interdisciplinary Musicology (CIM14), celebrada els dies 4 a 6 de desembre de 2014 a Berlín, Alemanya., Most of the current musicological knowledge is present in printed books and manuscripts. In the last years greats efforts have been done in order to digitize and make available these documents in form of Digital Libraries. However, digital documents are mainly stored as raw text, with no more structure than indexes and some metadata. Therefore, implicit knowledge contained in text is not understandable by computers and cannot be processed like that. Automatic processing of text documents may help musicologists in several ways, such as improving navigation through a library, discovering hidden knowledge, accelerating tedious tasks, etc. To apply these techniques to a Digital Library, the information contained in documents should be carefully structured and semantically annotated. Information Extraction is a discipline of computer science focused on the extraction of structured information from unstructured text sources. We propose a method to automatically extract meaningful knowledge from documents present in Digital Musical Document Libraries, by using Information Extraction techniques. Our method has two main steps. First, relevant named entities (e.g. composers, organizations, places, etc.) are identified in the text. Second, words between these entities are syntactically and semantically analyzed to understand the relationship between them. Finally, the extracted knowledge is represented in a machine-readable format as a knowledge graph, where entities are represented as nodes, and relations as edges. The resulting knowledge representation is finally visualized as an interactive graph. With the proposed information visualization, users may go from one document to another by browsing the knowledge graph. We tested our method with a subset of artist biographies present in the Grove Music Online., This research was funded by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Transcription and recognition of syllable based percussion patterns: the case of Beijing Opera

  • Srinivasamurthy, Ajay
  • Caro Repetto, Rafael
  • Sundar, Harshavardhan
  • Serra, Xavier
In many cultures of the world, traditional percussion music/nuses mnemonic syllables that are representative of the timbres/nof instruments. These syllables are orally transmitted/nand often provide a language for percussion in those music/ncultures. Percussion patterns in these cultures thus have/na well defined representation in the form of these syllables,/nwhich can be utilized in several computational percussion/npattern analysis tasks. We explore a connected word/nspeech recognition based framework that can effectively/nutilize the syllabic representation for automatic transcription/nand recognition of audio percussion patterns. In particular,/nwe consider the case of Beijing opera and present/na syllable level hidden markov model (HMM) based system/nfor transcription and classification of percussion patterns./nThe encouraging classification results on a representative/ndataset of Beijing opera percussion patterns supports/nour approach and provides further insights on the utility of/nthese syllables for computational description of percussion/npatterns., This work is partly supported by the European Research/nCouncil under the European Union’s Seventh Framework/nProgram, as a part of the CompMusic project (ERC grant/nagreement 267583)
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Study of the similarity between linguistic tones and melodic pitch contours in Beijing Opera singing

  • Zhang, Shuo
  • Caro Repetto, Rafael
  • Serra, Xavier
Comunicació presentada a la 15th International Society for Music Information Retrieval Conference (ISMIR 2014), celebrada els dies 27 a 31 d'octubre de 2014 a Taipei, Taiwan., Features of linguistic tone contours are important factors that shape the distinct melodic characteristics of different genres of Chinese opera. In Beijing opera, the presence of a two-dialectal tone system makes the tone-melody relationship more complex. In this paper, we propose a novel data-driven approach to analyze syllable-sized tone-pitch contour similarity in a corpus of Beijing Opera (381 arias) with statistical modeling and machine learning methods. A total number of 1,993 pitch contour units and attributes were extracted from a selection of 20 arias. We then build Smoothing Spline ANOVA models to compute matrixes of average melodic contour curves by tone category and other attributes. A set of machine learning and statistical analysis methods are applied to 30-point pitch contour vectors as well as dimensionality-reduced representations using Symbolic Aggregate approXimation(SAX). The results indicate an even mixture of shapes within all tone categories, with the absence of evidence for a predominant dialectal tone system in Beijing opera. We discuss the key methodological issues in melody-tone analysis and future work on pair-wise contour unit analysis., This research was funded by the European Research Council under the European Union Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A study of instrument-wise onset detection in Beijing Opera percussion ensembles

  • Tian, Mi
  • Srinivasamurthy, Ajay
  • Sandler, Mark
  • Serra, Xavier
Note onset detection and instrument recognition are two of the most investigated tasks in Music Information Retrieval (MIR). Various detection methods have been proposed in previous research for western music, with less focus on other music cultures of the world. In this paper, we focus on onset detection for percussion instruments in Beijing Opera, a major genre of Chinese traditional music. A dataset of individual audio samples of four primary percussion instruments is used to obtain the spectral bases for each instrument. With these bases, we separate the input percussion ensemble recordings into its spectral sources and their activations using a Non-negative Matrix Factorization (NMF) based algorithm. A simple onset detection conducted on each NMF activation presents satisfactory overall detection rates, and provides us valuable implications and suggestions for future development of drum transcription and percussion pattern analysis in Beijing Opera.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Improving melodic similarity in Indian art music using culture-specific melodic characteristics

  • Gulati, Sankalp
  • Serrà Julià, Joan
  • Serra, Xavier
Comunicació presentada a la 16th International Society for Music Information Retrieval Conference (ISMIR 2015), celebrada els dies 26 a 30 d'octubre de 2015 a Màlaga, Espanya., Detecting the occurrences of rags’ characteristic melodic phrases from polyphonic audio recordings is a fundamental task for the analysis and retrieval of Indian art music. We propose an abstraction process and a complexity weighting scheme which improve melodic similarity by exploiting specific melodic characteristics in this music. In addition, we propose a tetrachord normalization to handle transposed phrase occurrences. The melodic abstraction is based on the partial transcription of the steady regions in the melody, followed by a duration truncation step. The proposed complexity weighting accounts for the differences in the melodic complexities of the phrases, a crucial aspect known to distinguish phrases in Carnatic music. For evaluation we use over 5 hours of audio data comprising 625 annotated melodic phrases belonging to 10 different phrase categories. Results show that the proposed melodic abstraction and complexity weighting schemes significantly improve the phrase detection accuracy, and that tetrachord normalization is a successful strategy for dealing with transposed phrase occurrences in Carnatic music. In the future, it would be worthwhile to explore the applicability of the proposed approach to other melody dominant music traditions such as Flamenco, Beijing opera and Turkish Makam music., This work is partly supported by the European Research Council under the European Unions Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Intonation analysis of rāgas in Carnatic music

  • Koduri, Gopala Krishna
  • Ishwar, Vignesh
  • Serrà Julià, Joan
  • Serra, Xavier
Intonation is a fundamental music concept that has a special relevance in Indian art music. It is characteristic of a rga and key to the musical expression of the artist. Describing intonation is of importance to several music information retrieval tasks such as developing similarity measures based on rgas and artists. In this paper, we first assess rga intonation qualitatively by analysing varṇaṁs, a particular form of Carnatic music compositions. We then approach the task of automatically obtaining a compact representation of the intonation of a recording from its pitch track. We propose two approaches based on the parametrization of pitch-value distributions: performance pitch histograms, and context-based svara distributions obtained by categorizing pitch contours based on the melodic context. We evaluate both approaches on a large Carnatic music collection and discuss their merits and limitations. We finally go through different kinds of contextual information that can be obtained to further improve the two approaches., This research was partly funded by the European Research Council/nunder the European Union’s Seventh Framework/nProgram, as part of the CompMusic project (ERC grant agreement/n267583). J.S. acknowledges 2009-SGR-1434 fromGeneralitat/nde Catalunya, ICT-2011-8-318770 from the European Commission,/nJAEDOC069/2010 from CSIC, and European Social/nFunds.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Mining melodic patterns in large audio collections of Indian art music

  • Gulati, Sankalp
  • Serrà Julià, Joan
  • Ishwar, Vignesh
  • Serra, Xavier
Comunicació presentada a la 10th International Conference on Signal Image Technology and Internet Based Systems celebrada del 23 al 27 de novembre de 2014 a Marràqueix (Marroc)., Discovery of repeating structures in music is fundamental to its analysis, understanding and interpretation. We present a data-driven approach for the discovery of shorttime melodic patterns in large collections of Indian art music. The approach first discovers melodic patterns within an audio recording and subsequently searches for their repetitions in the entire music collection. We compute similarity between melodic patterns using dynamic time warping (DTW). Furthermore, we investigate four different variants of the DTW cost function for rank refinement of the obtained results. The music collection used in this study comprises 1,764 audio recordings with a total duration of 365 hours. Over 13 trillion DTW distance computations are done for the entire dataset. Due to the computational complexity of the task, different lower bounding and early abandoning techniques are applied during DTW distance computation. An evaluation based on expert feedback on a subset of the dataset shows that the discovered melodic patterns are musically relevant. Several musically interesting relationships are discovered, yielding further scope for establishing novel similarity measures based on melodic patterns. The discovered melodic patterns can further be used in challenging computational tasks such as automatic r¯aga recognition, composition identification and music recommendation., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583). J.S. acknowledges 2009-SGR- 1434 from Generalitat de Catalunya, ICT-2011-8-318770 from the European Commission, JAEDOC069/2010 from CSIC, and European Social Funds.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Acoustic scene classification by ensembling gradient boosting machine and convolutional neural networks

  • Fonseca, Eduardo
  • Gong, Rong
  • Bogdanov, Dmitry
  • Slizovskaia, Olga
  • Gómez Gutiérrez, Emilia, 1975-
  • Serra, Xavier
Comunicació presentada al Detection and Classification of Acoustic Scenes and Events 2017 Workshop (DCASE2017), celebrat el dia 16 de novembre de 2017 a Munic, Alemanya., This work describes our contribution to the acoustic scene classifi- cation task of the DCASE 2017 challenge. We propose a system that consists of the ensemble of two methods of different nature: a feature engineering approach, where a collection of hand-crafted features is input to a Gradient Boosting Machine, and another approach based on learning representations from data, where log-scaled melspectrograms are input to a Convolutional Neural Network. This CNN is designed with multiple filter shapes in the first layer. We use a simple late fusion strategy to combine both methods. We report classification accuracy of each method alone and the ensemble system on the provided cross-validation setup of TUT Acoustic Scenes 2017 dataset. The proposed system outperforms each of its component methods and improves the provided baseline system by 8.2%., This work is partially supported by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 688382 “AudioCommons”, and the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Landmark detection in Hindustani music melodies

  • Gulati, Sankalp
  • Serrà Julià, Joan
  • Ganguli, Kaustuv Kanti
  • Serra, Xavier
Comunicació presentada a la International Computer Music Conference, ICMC/SMC, celebrada a Atenes (Grècia) del 14 al 20 de setembre de 2014., Musical melodies contain hierarchically organized events, where some events are more salient than others, acting as melodic landmarks. In Hindustani music melodies, an important landmark is the occurrence of a nyas. Occurrence of nyas is crucial to build and sustain the format of a r¯ag and mark the boundaries of melodic motifs. Detection of ny¯as segments is relevant to tasks such as melody segmentation, motif discovery and r¯ag recognition. However, detection of ny¯as segments is challenging as these segments do not follow explicit set of rules in terms of segment length, contour characteristics, and melodic context. In this paper we propose a method for the automatic detection of nyas segments in Hindustani music melodies. It consists of two main steps: a segmentation step that incorporates domain knowledge in order to facilitate the placement of nyas boundaries, and a segment classification step that is based on a series of musically motivated pitch contour features. The proposed method obtains significant accuracies for a heterogeneous data set of 20 audio music recordings containing 1257 nyas svar occurrences and total duration of 1.5 hours. Further, we show that the proposed segmentation strategy significantly improves over a classical piece-wise linear segmentation approach., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583). J.S. acknowledges 2009-SGR-1434 from Generalitat de Catalunya, ICT-2011-8-318770 from the European Commission, JAEDOC069/2010 from CSIC, and European Social Funds.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

On the use of note onsets for improved lyrics-to-audio alignment in Turkish Makam music

  • Dzhambazov, Georgi Bogomilov
  • Srinivasamurthy, Ajay
  • Sentürk, Sertan
  • Serra, Xavier
Comunicació presentada a la 17th International Society for Music Information Retrieval Conference (ISMIR 2016), celebrada els dies 7 a 11 d'agost de 2016 a Nova York, EUA., Lyrics-to-audio alignment aims to automatically match given lyrics and musical audio. In this work we extend a state of the art approach for lyrics-to-audio alignment with information about note onsets. In particular, we consider the fact that transition to next lyrics syllable usually implies transition to a new musical note. To this end we formulate rules that guide the transition between consecutive phonemes when a note onset is present. These rules are incorporated into the transition matrix of a variable-time hidden Markov model (VTHMM) phonetic recognizer based on MFCCs. An estimated melodic contour is input to an automatic note transcription algorithm, from which the note onsets are derived. The proposed approach is evaluated on 12 a cappella audio recordings of Turkish Makam music using a phrase-level accuracy measure. Evaluation of the alignment is also presented on a polyphonic version of the dataset in order to assess how degradation in the extracted onsets affects performance. Results show that the proposed model outperforms a baseline approach unaware of onset transition rules. To the best of our knowledge, this is the one of the first approaches tackling lyrics tracking, which combines timbral features with a melodic feature in the alignment process itself., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583) and partly by the AGAUR research grant. We acknowledge as well financial support from the Spanish Ministry of Economy and Competitiveness, through the ”María de Maeztu” Programme for Centres/Units of Excellence in R&D” (MDM-2015-0502).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Audio clip classification using social tags and the effect of tag expansion

  • Font Corbera, Frederic
  • Serrà Julià, Joan
  • Serra, Xavier
Comunicació presentada a la 53rd International Conference: Semantic audio, celebrada els dies 27 a 29 de gener de 2014 a Londres, Regne Unit., Methods for automatic sound and music classification are of great value when trying to organise the large amounts of unstructured, user-contributed audio content uploaded to online sharing platforms. Currently, most of these methods are based on the audio signal, leaving the exploitation of users’ annotations or other contextual data rather unexplored. In this paper, we describe a method for the automatic classification of audio clips which is solely based on user-supplied tags. As a novelty, the method includes a tag expansion step for increasing classification accuracy when audio clips are scarcely tagged. Our results suggest that very high accuracies can be achieved in tag-based audio classification (even for poorly or badly annotated clips), and that the proposed tag expansion step can, in some cases, significantly increase classification performance. We are interested in the use of the described classification method as a first step for tailoring assistive tagging systems to the particularities of different audio categories, and as a way to improve the overall quality of online user annotations., This work has been supported by BES-2010-037309 FPI from the Spanish Ministry of Science and Innovation (TIN2009-14247-C02-01; F.F.), 2009-SGR-1434 from Generalitat de Catalunya (J.S.), JAEDOC069/2010 from Consejo Superior de Investigaciones Cient´ıficas (J.S.), ICT-2011-8-318770 from the European Commission (J.S.), and FP7-2007-2013 / ERC grant agreement 267583 (CompMusic).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Audio feature extraction for exploring Turkish makam music

  • Atli, Hasan Sercan
  • Uyar, Burak
  • Sentürk, Sertan
  • Bozkurt, Baris
  • Serra, Xavier
Comunicació presentada a: ATMM 2014 The 3rd International Conference on Audio Technologies for Music and Media, celebrat del 12 al 14 de Novembre de 2014 a Ankara i Istanbul, Turquia., For Turkish makam music, there exist several analysis tools which generally use only the audio as the input to extract the features of the audio. This study aims at extending such approach by using additional features such as scores, editorial metadata and the knowledge about the music. In this paper, the existing algorithms for similar research, the improvements we apply to the existing audio feature extraction tools and some potential topics for audio feature extraction of Turkish makam music are explained. For the improvements, we make use of the Turkish makam music corpus and the culture specific knowledge. We also present a web-based platform, Dunya, where the output of our research, such as pitch histograms, melodic progressions and segmentation information will be used to explore a collection of audio recordings of Turkish makam music., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Applause identification and its relevance to archival of Carnatic music

  • Sarala, Padi
  • Ishwar, Vignesh
  • Bellur, Ashwin
  • Murthy, Hema A.
Comunicació presentada al 2nd CompMusic Workshop, celebrat els dies 12 i 13 de juliol de 2012 a Istanbul (Turquia), organitzat per CompMusic., A Carnatic music concert is made up of a sequence of pieces, where each piece corresponds to a particular genre and ra¯aga (melody). Unlike a western music concert, the artist may be applauded intra-performance /ninter-performance. Most Carnatic music that is archived today correspond to a single audio recordings of entire concerts./nThe purpose of this paper is to segment single audio recordings into a sequence of pieces using the/ncharacteristic features of applause and music. Spectral flux, spectral entropy change quite significantly from music to applause and vice-versa. The characteristics of these features for a subset of concerts was studied. A threshold based approach was used to segment the pieces into music fragments and applauses. Preliminary results/non recordings 19 concerts from matched microphones show that the EER is about 17% for a resolution of 0.25 seconds. Further, a parameter called CUSUM is estimated/nfor the applause regions. The CUSUM values determine the strength of the applause. The CUSUM is used to characterise the highlights of a concert., This research was partly funded by the European Research Council under the European Unions Seventh Framework Program, as part of the CompMusic project (ERC grant/nagreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Classification of indian classical vocal styles from melodic contours

  • Vidwans, Amruta
  • Ganguli, Kaustuv Kanti
  • Rao, Preeti
A prominent categorization of Indian classical music is the Hindustani and Carnatic traditions, the two styles/nhaving evolved under distinctly different historical and/ncultural influences. Both styles are grounded in the melodic/nand rhythmic framework of raga and tala. The styles differ along dimensions such as instrumentation,/naesthetics and voice production. In particular, Carnatic music is perceived as being more ornamented. The hypothesis/nthat style distinctions are embedded in the melodic contour is validated via subjective classification tests. Melodic features representing the distinctive characteristics/nare extracted from the audio. Previous work based on the extent of stable pitch regions is supported by measurements of musicians’ annotations of stable notes. Further, a new feature is introduced that captures the/npresence of specific pitch modulations characteristic of/nornamentation in Indian classical music. The combined features show high classification accuracy on a database of vocal music of prominent artistes. The misclassifications are seen to match actual listener confusions.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A two-stage approach for tonic identification in indian art music

  • Gulati, Sankalp
  • Salamon, Justin
  • Serra, Xavier
In this paper we propose a new approach for tonic identification in Indian art music and present a proposal for a/ncomplete iterative system for the same. Our method splits the task of tonic pitch identification into two stages. In the first stage, which is applicable to both vocal and instrumental music, we perform a multi-pitch analysis of the audio signal to identify the tonic pitch-class. Multi-pitch analysis/nallows us to take advantage of the drone sound, which constantly/nreinforces the tonic. In the second stage we estimate the octave in which the tonic of the singer lies and is thus/nneeded only for the vocal performances. We analyse the predominant melody sung by the lead performer in order to establish the tonic octave. Both stages are individually evaluated on a sizable music collection and are shown to/nobtain a good accuracy. We also discuss the types of errors made by the method./nFurther, we present a proposal for a system that aims to incrementally utilize all the available data, both audio and metadata in order to identify the tonic pitch. It produces a tonic estimate and a confidence value, and is iterative in nature. At each iteration, more data is fed into the system/nuntil the confidence value for the identified tonic is above a defined threshold. Rather than obtain high overall accuracy for our complete database, ultimately our goal is to develop a system which obtains very high accuracy on a subset of the database with maximum confidence., This research was funded by the European Research Council under the European Union's Seventh Framework Programme/n(FP7/2007-2013) / ERC grant agreement 267583 (CompMusic).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Automatic transcription of Turkish makam music

  • Benetos, Emmanouil
  • Holzapfel, Andre
In this paper we propose an automatic system for transcribing/nmakam music of Turkey. We document the specific/ntraits of this music that deviate from properties that/nwere targeted by transcription tools so far and we compile/na dataset of makam recordings along with aligned microtonal/nground-truth. An existing multi-pitch detection algorithm/nis adapted for transcribing music in 20 cent resolution,/nand the final transcription is centered around the/ntonic frequency of the recording. Evaluation metrics for/ntranscribing microtonal music are utilized and results show/nthat transcription of Turkish makam music in e.g. an interactive/ntranscription software is feasible using the current/nstate-of-the-art., This work is partly supported by the European/nResearch Council under the European Union’s Seventh/nFramework Program, as part of the CompMusic project/n(ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Predicting pairwise pitch contour relations based on linguistic tone information in Beijing opera singing

  • Zhang, Shuo
  • Caro Repetto, Rafael
  • Serra, Xavier
Comunicació presentada a la 16th International Society for Music Information Retrieval Conference (ISMIR 2015), celebrada els dies 26 a 30 d'octubre de 2015 a Màlaga, Espanya., The similarity between linguistic tones and melodic pitch contours in Beijing Opera can be captured either by the contour shape of single syllable units, or by the pairwise pitch height relations in adjacent syllable units. In this paper, we investigate the latter problem with a novel machine learning approach, using techniques from time series data mining. Approximately 1300 pairwise contour segments are extracted from a selection of 20 arias. We then formulate the problem as a supervised machinelearning task of predicting types of pairwise melodic relations based on linguistic tone information. The results give a comparative view of fixed and mixed-effects models that achieved around 70% of maximum accuracy. We discuss the superiority of the current method to that of the unsupervised learning in single-syllable-unit contour analysis of similarity in Beijing Opera., This research was funded by the European Research Council under the European Union Seventh Framework Program, as part of the CompMusic Project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Extracting semantic information from an online Carnatic music forum

  • Sordo, Mohamed
  • Serrà Julià, Joan
  • Koduri, Gopala Krishna
  • Serra, Xavier
By mining user-generated text content we can obtain musicrelated/ninformation that could not otherwise be extracted/nfrom audio signals or symbolic score representations. In/nthis paper we propose a methodology for extracting/nmusic-related semantic information from an online/ndiscussion forum, rasikas.org, dedicated to the Carnatic/nmusic tradition. We first define a dictionary of relevant/nterms within categories such as raagas, taalas, performers,/ncomposers, and instruments, and create a complex network/nrepresentation by matching such dictionary against the forum/nposts. This network representation is used to identify/npopular terms within the forum, as well as relevant/nco-occurrences and semantic relationships. This way, for/ninstance, we are able to learn the instrument played by a/nperformer with 95% accuracy, to discover the confusion/nbetween two raagas with different naming conventions, or/nto infer semantic relationships regarding lineage or musical/ninfluence. This contribution is a first step towards the/nautomatic creation of ontologies for specific musical cultures., This research was partly funded by the European Research/nCouncil under the European Union’s Seventh Framework/nProgram, as part of the CompMusic project (ERC/ngrant agreement 267583). Joan Serrà acknowledges AEDOC069/2010 from Consejo Superior de Investigaciones/nCientíficas, 2009-SGR-1434 from Generalitat de Catalunya,/nTIN2009-13692-C03-01 from the Spanish Government,/nand EU Feder Funds.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A method for tonic frequency identification of Turkish Makam music recordings

  • Atli, Hasan Sercan
  • Bozkurt, Baris
  • Sentürk, Sertan
Comunicació presentada al 5th International Workshop on Folk Music Analysis, celebrat a Paris (França) els dies 10 a 12 de juny de 2015., Karar is the final tone in the Turkish makam music performances. The karar frequency, hence the concert pitch varies among performances due to the existence of many diapasons in Turkish makam music, instead of a single standard one. Correct estimation of the karar frequency is critical for many computational tasks such as tuning analysis, audo-score alignment and automatic trascription. We present a new karar frequency identification method that is based on detecting the last note in the recording and estimating its frequency. The method is applied on two large datasets of Turkish makam music recordings and shown to outperform the state-of-the-art., This work is supported by the European Research Council under the European Union's Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 26758).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Assessing the tuning of sung indian classical music

  • Serrà Julià, Joan
  • Koduri, Gopala Krishna
  • Miron, Marius
  • Serra, Xavier
The issue of tuning in Indian classical music has been, historically,/na matter of theoretical debate. In this paper, we/nstudy its contemporary practice in sung performances of Carnatic/nand Hindustani music following an empiric and quantitative/napproach. To do so, we select stable fundamental frequencies,/nestimated via a standard algorithm, and construct/ninterval histograms from a pool of recordings. We then compare/nsuch histograms against the ones obtained for different/nmusic sources and against the theoretical values derived/nfrom 12-note just intonation and equal temperament. Our/nresults evidence that the tunings in Carnatic and Hindustani/nmusic differ, the former tending to a just intonation system/nand the latter having much equal-tempered influences. Carnatic/nmusic also presents signs of a more continuous distribution/nof pitches. Further subdivisions of the octave are/npartially investigated, finding no strong evidence of them., The research leading to these results/nhas received funding from the European Research Council/nunder the European Union’s Seventh Framework Programme/n(FP7/2007-2013) / ERC grant agreement 267583/n(CompMusic).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Essentia: an audio analysis library for music information retrieval

  • Bogdanov, Dmitry
  • Wack, Nicolas
  • Gómez Gutiérrez, Emilia, 1975-
  • Gulati, Sankalp
  • Herrera Boyer, Perfecto, 1964-
  • Mayor, Oscar
  • Roma Trepat, Gerard
  • Salamon, Justin
  • Zapata González, José Ricardo
  • Serra, Xavier
Comunicació presentada a la 14th International Society for Music Information Retrieval Conference, celebrada a Curitiba (Brasil) els dies 4 a 8 de novembre de 2013., We present Essentia 2.0, an open-source C++ library for audio analysis and audio-based music information retrieval released under the Affero GPL license. It contains an extensive collection of reusable algorithms which implement audio input/output functionality, standard digital signal processing blocks, statistical characterization of data, and a large set of spectral, temporal, tonal and high-level music descriptors. The library is also wrapped in Python and includes a number of predefined executable extractors for the available music descriptors, which facilitates its use for fast prototyping and allows setting up research experiments very rapidly. Furthermore, it includes a Vamp plugin to be used with Sonic Visualiser for visualization purposes. The library is cross-platform and currently supports Linux, Mac OS X, and Windows systems. Essentia is designed with a focus on the robustness of the provided music descriptors and is optimized in terms of the computational cost of the algorithms. The provided functionality, specifically the music descriptors included in-the-box and signal processing algorithms, is easily expandable and allows for both research experiments and development of large-scale industrial applications., The work on Essentia has been partially funded by the PHAROS (EUIP, IST-2006-045035), Buscamedia (CEN-20091026), CANTATA (ITEA 05010, FIT-350300-2006-33), SIGMUS (TIN2012-36650), CompMusic (ERC 267583), and TECNIO (TECCIT12-1-0003) projects.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Analysis of the impact of a tag recommendation system in a real-world folksonomy

  • Font Corbera, Frederic
  • Serrà Julià, Joan
  • Serra, Xavier
Collaborative tagging systems have emerged as a successful solution for annotating contributed resources to online sharing platforms, facilitating searching, browsing, and organizing their contents. To aid users in the annotation process, several tag recommendation methods have been proposed. It has been repeatedly hypothesized that these methods should contribute to improving annotation quality and reducing the cost of the annotation process. It has been also hypothesized that these methods should contribute to the consolidation of the vocabulary of collaborative tagging systems. However, to date, no empirical and quantitative result supports these hypotheses. In this work, we deeply analyze the impact of a tag recommendation system in the folksonomy of Freesound, a real-world and large-scale online sound sharing platform. Our results suggest that tag recommendation effectively increases vocabulary sharing among users of the platform. In addition, tag recommendation is shown to contribute to the convergence of the vocabulary as well as to a partial increase in the quality of annotations. However, according to our analysis, the cost of the annotation process does not seem to be effectively reduced. Our work is relevant to increase our understanding about the nature of tag recommendation systems and points to future directions for the further development of those systems and their analysis., This work has been supported by BES-2010-037309 FPI from the Spanish Ministry of Science and Innovation, TIN2009-14247-C02-01 from the Spanish government (F.F.), 2009-SGR-1434 from Generalitat de Catalunya (J.S.), JAEDOC069/2010 from CSIC (J.S.), FSE2007-2013 E. U. Social funds (J.S.), and FP7-2007-2013/ERC grant agreement 267583 (CompMusic; F.F. and X.S.).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Timbre analysis of music audio signals with convolutional neural networks

  • Pons Puig, Jordi
  • Slizovskaia, Olga
  • Gómez Gutiérrez, Emilia, 1975-
  • Serra, Xavier
Comunicació presentada a la EUSIPCO 2017: 25th European Signal Processing Conference, celebrada els dies 28 d'agost a 2 de setembre de 2017 a Kos, Grècia., The focus of this work is to study how to efficiently tailor Convolutional Neural Networks (CNNs) towards learning timbre representations from log-mel magnitude spectrograms. We first review the trends when designing CNN architectures. Through this literature overview we discuss which are the crucial points to consider for efficiently learning timbre representations using CNNs. From this discussion we propose a design strategy meant to capture the relevant time-frequency contexts for learning timbre, which permits using domain knowledge for designing architectures. In addition, one of our main goals is to design efficient CNN architectures – what reduces the risk of these models to over-fit, since CNNs’ number of parameters is minimized. Several architectures based on the design principles we propose are successfully assessed for different research tasks related to timbre: singing voice phoneme classification, musical instrument recognition and music auto-tagging., This work is partially supported by: the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502), the CompMusic project (ERC grant agreement 267583) and the CASAS Spanish research project (TIN2015-70816-R).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Score-informed syllable segmentation for Jingju a cappella singing voice with Mel-frequency intensity profiles

  • Gong, Rong
  • Obin, Nicolas
  • Dzhambazov, Georgi Bogomilov
  • Serra, Xavier
Comunicació presentada al International Workshop on Folk Music Analysis, celebrat del 14 al 16 de juny a Màlaga, Espanya., This paper introduces a new unsupervised and score-informed method for the segmentation of singing voice into syllables. The main idea of the proposed method is to detect the syllable onset on a probability density function by incorporating a priori syllable duration derived from the score. Firstly, intensity profiles are used to exploit the characteristics of singing voice depending on the Mel-frequency regions. Then, the syllable onset probability density function is obtained by selecting candidates over the intensity profiles and weighted for the purpose of emphasizing the onset regions. Finally, the syllable duration distribution shaped by the score is incorporated into Viterbi decoding to determine the optimal sequence of onset time positions. The proposed method outperforms conventional methods for the segmentation of syllable on a jingju (also known as Peking or Beijing opera) a cappella dataset. An analysis is conducted on precision errors to provide direction for future improvement., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grand agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A method for extracting semantic information from on-line art music discussion forums

  • Sordo, Mohamed
  • Serrà Julià, Joan
  • Koduri, Gopala Krishna
  • Serra, Xavier
Comunicació presentada al 2nd CompMusic Workshop, celebrat els dies 12 i 13 de juliol de 2012 a Istanbul (Turquia), organitzat per CompMusic., In this paper a method for extracting semantic information/nfrom online music discussion forums is proposed. The semantic relations are inferred from the co-occurrence of musical concepts in forum posts, using network analysis. The method starts by defining a dictionary of common music terms in an art music tradition. Then, it creates a complex network representation of the online forum by matching/nsuch dictionary against the forum posts. Once the complex network is built we can study different network measures, including node relevance, node co-occurrence and/nterm relations via semantically connecting words. Moreover, we can detect communities of concepts inside the forum posts. The rationale is that some music terms are more related to each other than to other terms. All in all, this methodology allows us to obtain meaningful and relevant/ninformation from forum discussions., This research was partly funded by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583). Joan Serr`a acknowledges 2009-SGR-1434 from Generalitat de Catalunya and JAEDOC069/2010 from Consejo Superior de Investigaciones/nCientíficas.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

ESSENTIA: an open-source library for sound and music analysis

  • Bogdanov, Dmitry
  • Wack, Nicolas
  • Gómez Gutiérrez, Emilia, 1975-
  • Gulati, Sankalp
  • Herrera Boyer, Perfecto, 1964-
  • Mayor, Oscar
  • Roma Trepat, Gerard
  • Salamon, Justin
  • Zapata González, José Ricardo
  • Serra, Xavier
Comunicació presentada a la 21st ACM international conference on Multimedia, celebrada els dies 21 a 25 d'octubre de 2013 a Barcelona, Espanya., We present Essentia 2.0, an open-source C++ library for audio analysis and audio-based music information retrieval released under the Affero GPL license. It contains an extensive collection of reusable algorithms which implement audio input/output functionality, standard digital signal processing blocks, statistical characterization of data, and a large set of spectral, temporal, tonal and high-level music descriptors. The library is also wrapped in Python and includes a number of predefined executable extractors for the available music descriptors, which facilitates its use for fast prototyping and allows setting up research experiments very rapidly. Furthermore, it includes a Vamp plugin to be used with Sonic Visualiser for visualization purposes. The library is cross-platform and currently supports Linux, Mac OS X, and Windows systems. Essentia is designed with a focus on the robustness of the provided music descriptors and is optimized in terms of the computational cost of the algorithms. The provided functionality, specifically the music descriptors included in-the-box and signal processing algorithms, is easily expandable and allows for both research experiments and development of large-scale industrial applications., The work on Essentia has been partially funded by the PHAROS (EU-IP, IST-2006-045035), CANTATA (ITEA 05010, FIT-350300-2006-33), Buscamedia (CEN-20091026), SIGMUS (TIN2012-36650), CompMusic (ERC 267583), and TECNIO (TECCIT12-1-0003) projects.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Characterization of intonation in Karṇāṭaka music by parametrizing context-based Svara Distributions

  • Koduri, Gopala Krishna
  • Serrà Julià, Joan
  • Serra, Xavier
Intonation is a fundamental music concept that has a special relevance in Indian art music. It is characteristic of the rāga and intrinsic to the musical expression of the performer. Describing intonation is of importance to several information retrieval tasks like the development of rāga and artist similarity measures. In our previous work, we proposed a compact representation of intonation based on the parametrization of the pitch histogram of a performance and demonstrated the usefulness of this representation through an explorative rāga recognition task in which we classified 42 vocal performances belonging to 3 rāgas using parameters of a single svara. In this paper, we extend this representation to employ context-based svara distributions, which are obtained with a different approach to find the pitches belonging to each svara. We quantitatively compare this method to our previous one, discuss the advantages, and the necessary melodic analysis to be carried out in future., This research was partly funded by the European Research Council under the European Union’s Seventh Framework/nProgram, as part of the CompMusic project (ERC grant agreement 267583). JS acknowledges JAEDOC069/2010/nfrom Consejo Superior de Investigaciones Científcas and/n2009-SGR-1434 from Generalitat de Catalunya.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Data gathering for a culture specific approach in MIR

  • Serra, Xavier
In this paper we describe the data gathering work done within a large research project, CompMusic, which emphasizes a culture specific approach in the automatic description of several world music repertoires. Currently we are focusing on the Hindustani (North India), Carnatic (South India) and Turkish-makam (Turkey) music traditions. The selection and organization of the data to be processed for the characterization of each of these traditions is of the utmost importance., The CompMusic project has received funding from the European Research Council under the European Union’s Seventh Framework Program (FP7/2007-2013) / ERC grant agreement 267583.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

A microtonal tuning method: test & discussion on the use of microtonal intervals for the performance of traditional Turkish music

  • Tamer, Yahya Burak
  • Bozkurt, Baris
Background in music computing: An automatic method for tuning analysis of recordings was presented in (Bozkurt 2008). Data collected from this method have been used to develop a new microtonal tuner system implemented in Max MSP and Java (Tamer and Bozkurt 2013). The distinctive quality of the tuning method (previously demonstrated in (Bozkurt 2012)) is the use of a recording as reference instead of theoretical presets. During the process, fundamental frequency analysis is performed and pitch distributions are obtained for the selected recordings. This data is visualized via the tuner interface along with the real-time frequency estimate from the input signal to realize the tuning operation. Background in music performance: Electronic tuner devices are widely used by musicians. However these devices offer only a limited number of theoretical presets and equal temperament is the primary one. Lack of consistency between tuning theory and its practice of Turkish makam music has been addressed in a number of studies such as (Ay and Akkal 2008; Bozkurt et al. 2009). In the general traditional Turkish music concept, when microtonal intervals are used, the question of how musicians interpret the theory and apply it into their tuning practices arises. Aims: We investigate the relevance of tuning practices applied by three selected musicians who perform traditional Turkish music with the authentic intervals used by master musicians in selected recordings. We present the microtonal tuner application in order to provide a basic research tool for musicians and ethnomusicologists. Main contribution: The manuscript presents the user tests carried out with three professional musicians who perform traditional Turkish music and discusses the outcomes. A semistructured interview method is applied in order to test the process with the musicians. The tests contain comparisons of performances with and without the aid of the microtonal tuner application, providing insight to the musicians’ regular tuning practices and discussing further possibilities on the use of the application. The tests include recordings by well-known Turkish master musicians who perform microtonal music as Turkish makam music (one example) and Turkish folk music (three examples). Implications: By offering a direct access to any tuning system used by a master musician that has been captured in a recording, the application presented in this paper is suitable to serve as a computational ethnomusicology (CE) tool, thus has the potential to assist in ethnomusicological research. (Tzanetakis et al. 2007)., This work was funded in part by the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013) / ERC grant agreement 267583 (CompMusic).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Computational analysis of Turkish makam music: review of state-of-the-art and challenges

  • Bozkurt, Baris
  • Ayangil, Ruhi
  • Holzapfel, Andre
This text targets a review of the computational analysis literature for Turkish makam music, discussing in detail the challenges involved and presenting a perspective for further studies. For that purpose, the basic concepts of Turkish makam music and the description of melodic, rhythmic and timbral aspects are considered in detail. Studies on tuning analysis, automatic transcription, automatic melodic analysis, automatic makam and usul detection are reviewed. Technological and data resource needs for further advancement are discussed and available sources are presented., This work was funded by the European Research Council under the European Union’s Seventh Framework Pro-gramme (FP7/2007–2013)/ERC grant agreement 267583 (CompMusic).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Akshara transcription of mrudangam strokes in carnatic music

  • Kuriakose, Jom
  • Kumar, J Chaitanya
  • Sarala, Padi
  • Murthy, Hema A.
  • Sivaraman, Umayalpuram K
Percussion instruments play a significant role in/nCarnatic music concerts. The percussion artist enjoys a great/ndegree of freedom in improvising within the defined tala/nstructure/nof a composition. The objective of this paper is to transcribe the/nimprovisations, treating the percussion strokes as syllables or/naksharas./nOnset detection is performed to segment the waveform at/neach/nakshara/n. Using the transcriptions from the training data,/na three-state Hidden Markov Model is built for each/nakshara/n./nThe language model is derived from the training data. Testing/nis also performed isolated style using onset detection to segment/nthe phrase, and the language model to correct the transcription./nTranscription is performed on both concert recordings and studio/nrecordings. This technique yields upto/n≈/n96%/naccuracy on studio/nrecordings and/n≈/n76%/naccuracy for concert recordings./nAs the mrudangam/n1/nis an instrument that is based on tonic;/ntonic normalised features, namely, Cent Filterbank Cepstral/ncoefficients are used. It is shown that tonic normalisation helps/nin transcription across different tonics., This research was partly funded by the European Research/nCouncil under the European Unions Seventh Framework Pro-/ngram, as part of the CompMusic project (ERC grant agreement/n267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

An open corpus for the computational research of Arab-Andalusian music

  • Caro Repetto, Rafael
  • Pretto, Niccolò
  • Chaachoo, Amin
  • Bozkurt, Baris
  • Serra, Xavier
Comunicació presentada a: the 5th International Conference on Digital Libraries for Musicology (DLfM 2018), satèl·lit de l'ISMIR, celebrat el dia 28 de setembre de 2018 a París, França., In the medieval Islamic territories of the Iberian Peninsula known as Al-Andalus a unique style of music was formed combining local practices with Arab sensibilities. After the fall of the last Andalusian kingdom, this classical repertoire has been preserved to the present in North African countries. The idiosyncrasies of this repertoire, which combines musical traits from Western and Eastern Mediterranean traditions in orchestral and choral settings, as well as instrumental and vocal solos, deserves an in depth musicological study, that can benefit from computational tools for corpusdriven research. On the other hand, the characteristics of this music poses interesting challenges to MIR methods and therefore offer new research opportunities to this field. To address these topics, we present here the first complete release of the corpus for the research of the Moroccan tradition of Arab-Andalusian music built in the framework of the CompMusic project. The corpus comprises three data collections, namely audio recordings, music scores and lyrics, as well as related annotations and metadata. We also present a series of Jupyter Notebooks for browsing and retrieving data from the corpus. Both the corpus and notebooks are completely open to the research community., This work was carried out with the support of the Musical Bridges project, funded by RecerCaixa. The Andalusian Corpus was created by the CompMusic project, funded by the European Research Council under the European Union’s Seventh Framework Program (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Signal analysis of Ney performances

  • Özaslan, Tan Hakan
  • Serra, Xavier
  • Arcos, Josep Lluís
Ney is an end-blown flute which is mainly used for Makam music. Although from the beginning of 20th century a score representation based on extending the Western music/nis used, because of its rich articulation repertoire, actual/nNey music can not be totally represented by written score./nNey is still taught and transmitted orally in Turkey. Because/nof that the performance has a distinct and important/nrole in Ney music. Therefore signal analysis of ney performances/nis crucial for understanding the actual music./nAnother important aspect which is also a part of the performance/nis the articulations that performers apply. In Makam music in Turkey none of the articulations are taught even/nnamed by teachers. Articulations in Ney are valuable for understanding the real performance. Since articulations are not taught and their places are not marked in the score, the choice and character of the articulation is unique for each/nperformer which also makes each performance unique./nOur method analyzes audio files of well known Turkish Ney players. In order to obtain our analysis data, we analyzed audio files of 8 different performers vary from 1920/nto 2000.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Analysis of the folksonomy of freesound

  • Font Corbera, Frederic
  • Serra, Xavier
Comunicació presentada al 2nd CompMusic Workshop, celebrat els dies 12 i 13 de juliol de 2012 a Istanbul (Turquia), organitzat per CompMusic., User generated content shared in online communities is often described using collaborative tagging systems where users assign labels to content resources. As a result, a folksonomy emerges that relates a number of tags with the resources they label and the users that have used them. In this paper we analyze the folksonomy of Freesound, an online audio clip sharing site which contains more than two million users and 150,000 user-contributed sound samples/ncovering a wide variety of sounds. By following methodologies taken from similar studies, we compute some metrics that characterize the folksonomy both at the global/nlevel and at the tag level. In this manner, we are able to better/nunderstand the behavior of the folksonomy as a whole, and also obtain some indicators that can be used as metadata for describing tags themselves. We expect that such a methodology for characterizing folksonomies can be useful to support processes such as tag recommendation or automatic annotation of online resources., This work is partially supported by the European Research Council under the European Unions Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583), and by the Spanish Ministry of Science and Innovation under the BES-2010-037309 FPI grant for the TIN2009-14247-C02-01 DRIMS project.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Time-delayed melody surfaces for Rāga recognition

  • Gulati, Sankalp
  • Serrà Julià, Joan
  • Ganguli, Kaustuv Kanti
  • Sentürk, Sertan
  • Serra, Xavier
Comunicació presentada a la 17th International Society for Music Information Retrieval Conference (ISMIR 2016), celebrada els dies 7 a 11 d'agost de 2016 a Nova York, EUA., Raga is the melodic framework of Indian art music. It is a core concept used in composition, performance, organization, and pedagogy. Automatic raga recognition is thus a fundamental information retrieval task in Indian art music. In this paper, we propose the time-delayed melody surface (TDMS), a novel feature based on delay coordinates that captures the melodic outline of a raga. A TDMS describes both the tonal and the temporal characteristics of a melody, using only an estimation of the predominant pitch. Considering a simple k-nearest neighbor classifier, TDMSs outperform the state-of-the-art for raga recognition by a large margin. We obtain 98% accuracy on a Hindustani music dataset of 300 recordings and 30 ragas, and 87% accuracy on a Carnatic music dataset of 480 recordings and 40 ragas. TDMSs are simple to implement, fast to compute, and have a musically meaningful interpretation. Since the concepts and formulation behind the TDMS are generic and widely applicable, we envision its usage in other music traditions beyond Indian art music., This work is partly supported by the European Research Council under the European Unions Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Exploiting domain knowledge in music information research

  • Serra, Xavier
Comunicació presentada a la Sound and Music Computing Conference 2013, celebrada a Estocolm (Suècia) els dies 30 de juliol a 3 d'agost de 2013., Music Information Research (MIR) is a discipline that aims to understand and model music from an information processing perspective, but the successful approaches used in MIR are going beyond the traditional data processing methodologies. Most of the great advancements have been the result of combining engineering disciplines such as audio signal processing and machine learning with non-engineering disciplines such as music perception and music theory. One of the challenges in MIR is to automatically describe music audio signals, thus to develop methodologies to extract musically useful information from audio recordings. In this paper we claim that if we want to advance in this direction we should maximize the use of musical knowledge in all the steps of our research tasks. To support this claim we overview some of the work being carried out in CompMusic, a project that aims to analyze and automatically describe the music of several non-western music traditions., The CompMusic project is funded by the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013) / ERC grant agreement 267583.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Understanding the expressive functions of jingju metrical patterns through lyrics text mining

  • Zhang, Shuo
  • Caro Repetto, Rafael
  • Serra, Xavier
Comunicació presentada al MediaEval 2017 Workshop celebrat a Suzhou, Xina, del 23 al 27 d'octubre de 2017., The emotional content of jingju (aka Beijing or Peking opera) arias is conveyed through pre-defined metrical patterns known as banshi, each of them associated with a specific expressive function. In this paper, we first report the work on a comprehensive corpus of jingju lyrics that we built, suitable for text mining and text analysis in a data-driven framework. Utilizing this corpus, we propose a novel approach to study the expressive functions of banshi by applying text analysis techniques on lyrics. First we apply topic modeling techniques to jingju lyrics text documents grouped at different levels according to the banshi they are associated with. We then experiment with several different document vector representations of lyrics in a series of document classification experiments. The topic modeling results showed that sentiment polarity (positive or negative) is better distinguished between different shengqiang-banshi (a more fine grained partition of banshi) than banshi alone, and we are able to achieve high accuracy scores in classifying lyrics documents into different banshi categories. We discuss the technical and musicological implications and possible future improvements., This research is funded by the European Research Council under the European Union’s Seventh Framework Program (FP7/2007- 2013), as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Class-based tag recommendation and user-based evaluation in online audio clip sharing

  • Font Corbera, Frederic
  • Serrà Julià, Joan
  • Serra, Xavier
Online sharing platforms often rely on collaborative tagging systems for annotating content. In this way, users themselves annotate and describe the shared contents using textual labels, commonly called tags. These annotations typically suffer from a number of issues such as tag scarcity or ambiguous labelling. Hence, to minimise some of these issues, tag recommendation systems can be employed to suggest potentially relevant tags during the annotation process. In this work, we present a tag recommendation system and evaluate it in the context of an online platform for audio clip sharing. By exploiting domain-specific knowledge, the system we present is able to classify an audio clip among a number of predefined audio classes and to produce specific tag recommendations for the different classes. We perform an in-depth user-based evaluation of the recommendation method along with two baselines and a former version that we described in previous work. This user-based evaluation is further complemented with a prediction-based evaluation following standard information retrieval methodologies. Results show that the proposed tag recommendation method brings a statistically significant improvement over the previous method and the baselines. In addition, we report a number of findings based on the detailed analysis of user feedback provided during the evaluation process. The considered methods, when applied to real-world collaborative tagging systems, should serve the purpose of consolidating the tagging vocabulary and improving the quality of content annotations., This work has been supported by BES-2010-037309 FPI from the Spanish Ministry of Science and Innovation (TIN2009-14247-C02-01; F.F.), 2009-SGR-1434 from Generalitat de Catalunya (J.S.), JAEDOC069/2010 from CSIC (J.S.), ICT-2011-8-318770 from the European Commission (J.S.), and FP7-2007-2013/ERC Grant Agreement 267583 (CompMusic; F.F., X.S.).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A Multicultural approach in music information research

  • Serra, Xavier
Our information technologies do not respond to the world's/nmulticultural reality; in fact, we are imposing the paradigms/nof our market-driven western culture also on IT, thus facilitating/nthe access of a small part of the world’s information/nto a small part of the world's population. The current IT research/nefforts may even make it worse, and future IT will/naccentuate this information bias. Most IT research is being/ncarried out with a western centered approach and as a result,/nmost of our data models, cognition models, user models,/ninteraction models, ontologies, etc., are culturally biased./nThis fact is quite evident in music information research,/nsince, despite the world's richness in terms of musical/nculture, most research is centered on CDs and metadata/nof western commercial music. This is the motivation behind/na large and ambitious project funded by the European Research/nCouncil entitled "CompMusic: Computational Models/nfor the discovery of the world's music." In this paper we/npresent the ideas supporting this project, the challenges that/nwe want to work on, and the proposed approaches to tackle/nthese challenges., The CompMusic project has received funding from the/nEuropean Research Council under the European Union’s/nSeventh Framework Program (FP7/2007-2013) / ERC grant/nagreement 267583 and will run for five years starting on/nJuly 2011.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Informed automatic meter analysis of music recordings

  • Srinivasamurthy, Ajay
  • Holzapfel, Andre
  • Serra, Xavier
Comunicació presentada a la 18th International Society for Music Information Retrieval Conference (ISMIR 2017), celebrada els dies 23 a 27 d'octubre de 2017 a Suzhou, Xina., Automatic meter analysis aims to annotate a recording of a metered piece of music with its metrical structure. This analysis subsumes correct estimation of the type of meter, the tempo, and the alignment of the metrical structure with the music signal. Recently, Bayesian models have been successfully applied to several of meter analysis tasks, but depending on the musical context, meter analysis still poses significant challenges. In this paper, we investigate if there are benefits to automatic meter analysis from additional a priori information about the metrical structure of music. We explore informed automatic meter analysis, in which varying levels of prior information about the metrical structure of the music piece is available to analysis algorithms. We formulate different informed meter analysis tasks and discuss their practical applications, with a focus on Indian art music. We then adapt state of the art Bayesian meter analysis methods to these tasks and evaluate them on corpora of Indian art music. The experiments show that the use of additional information aids meter analysis and improves automatic meter analysis performance, with significant gains for analysis of downbeats., This work is partly supported by the European Research Council as part of the CompMusic project (ERC grant agreement 267583). Ajay Srinivasamurthy is currently with Idiap Research Institute, Martigny, Switzerland.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Creating a corpus of Jingju (Beijing Opera) music and possibilities for melodic analysis

  • Caro Repetto, Rafael
  • Serra, Xavier
Comunicació presentada a la 15th International Society for Music Information Retrieval Conference (ISMIR 2014), celebrada els dies 27 a 31 d'octubre de 2014 a Taipei, Taiwan., Jingju (Beijing opera) is a Chinese traditional performing art form in which theatrical and musical elements are in-timately combined. As an oral tradition, its musical di-mension is the result of the application of a series of pre-defined conventions and it offers unique concepts for mu-sicological research. Computational analyses of jingju music are still scarce, and only a few studies have dealt with it from an MIR perspective. In this paper we present the creation of a corpus of jingju music in the framework of the CompMusic project that is formed by audio, edito-rial metadata, lyrics and scores. We discuss the criteria followed for the acquisition of the data, describe the con-tent of the corpus, and evaluate its suitability for compu-tational and musicological research. We also identify several research problems that can take advantage of this corpus in the context of computational musicology, espe-cially for melodic analysis, and suggest approaches for future work., This research was funded by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Score informed tonic identification for Makam music of Turkey

  • Sentürk, Sertan
  • Gulati, Sankalp
  • Serra, Xavier
Tonic is a fundamental concept in many music traditions/nand its automatic identification should be relevant for establishing/nthe reference pitch when we analyse the melodic/ncontent of the music. In this paper, we present two methodologies/nfor the identification of the tonic in audio recordings/nof makam music of Turkey, both taking advantage/nof some score information. First, we compute a prominent/npitch and a audio kernel-density pitch class distribution/n(KPCD) from the audio recording. The peaks in the/nKPCD are selected as tonic candidates. The first method/ncomputes a score KPCD from the monophonic melody extracted/nfrom the score. Then, the audio KPCD is circularshifted/nwith respect to each tonic candidate and compared/nwith the score KPCD. The best matching shift indicates the/nestimated tonic. The second method extracts the monophonic/nmelody of the most repetitive section of the score./nNormalising the audio prominent pitch with respect to each/ntonic candidate, the method attempts to link the repetitive/nstructural element given in the score with the respective/ntime-intervals in the audio recording. The result producing/nthe most confident links marks the estimated tonic./nWe have tested the methods on a dataset of makam music/nof Turkey, achieving a very high accuracy (94.9%) with/nthe first method, and almost perfect identification (99.6%)/nwith the second method. We conclude that score informed/ntonic identification can be a useful first step in the computational/nanalysis (e.g. expressive analysis, intonation analysis,/naudio-score alignment) of music collections involving/nmelody-dominant content., This work is partly supported by the European Research/nCouncil under the European Union’s Seventh Framework/nProgram, as part of the CompMusic project (ERC grant/nagreement 267583).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Saraga: Open Datasets for Research on Indian Art Music

  • Srinivasamurthy, Ajay
  • Gulati, Sankalp
  • Caro Repetto, Rafael
  • Serra, Xavier
We introduce two large open data collections of Indian Art Music, both its Carnatic and Hindustani traditions, comprising audio from vocal concerts, editorial metadata, and time-aligned melody, rhythm, and structure annotations. Shared under Creative Commons licenses, they currently form the largest annotated data collections available for computational analysis of Indian Art Music. The collections are intended to provide audio and ground truth for several music information research tasks and large-scale data-driven analysis in musicological studies. A part of the Saraga Carnatic collection also has multitrack recordings, making it a valuable collection for research on melody extraction, source separation, automatic mixing, and performance analysis. We describe the tenets and the process of collection, annotation, and organization of the data. We provide easy access to the audio, metadata, and the annotations in the collections through an API, along with a companion website that has example scripts to facilitate access and use of the data. To sustain and grow the collections, we provide a mechanism for both the research and music community to contribute additional data and annotations to the collections. We also present applications with the collections for music education, understanding, exploration, and discovery., This work has been funded by the European Research Council under the European Union’s Seventh Framework Program (FP7/2007-2013)/ERC grant agreement 267583, The CompMusic Project (http://compmusic.upf.edu).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Modal analysis and transcription of strokes of the mridangam using non-negative matrix factorization

  • Anantapadmanabhan, Akshay
  • Bellur, Ashwin
  • Murthy, Hema A.
In this paper we use a Non-negative Matrix Factorization/n(NMF) based approach to analyze the strokes of the mri-/ndangam, a South Indian hand drum, in terms of the normal/nmodes of the instrument. Using NMF, a dictionary of spectral/nbasis vectors are first created for each of the modes of the/nmridangam. The composition of the strokes are then studied/nby projecting them along the direction of the modes using/nNMF. We then extend this knowledge of each stroke in terms/nof its basic modes to transcribe audio recordings. Hidden/nMarkov Models are adopted to learn the modal activations for/neach of the strokes of the mridangam, yielding up to/n88,40%/naccuracy during transcription., This research/nwas partly funded by the European Research Council under the/nEuropean/nUnions Seventh Framework Program, as part of the CompMusic project/n(ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Dunya: a system for browsing audio music collections exploiting cultural context

  • Porter, Alastair
  • Sordo, Mohamed
  • Serra, Xavier
Comunicació presentada a la 14th International Society for Music Information Retrieval Conference, celebrada a Curitiba (Brasil) els dies 4 a 8 de novembre de 2013., Music recommendation and discovery is an important MIR application with a strong impact in the music industry, but most music recommendation systems are still quite generic and without much musical knowledge. In this paper we present a web-based software application that lets users interact with an audio music collection through the use of musical concepts that are derived from a specific musical culture, in this case Carnatic music. The application includes a database containing information relevant to that music collection, such as audio recordings, editorial information, and metadata obtained from various sources. An analysis module extracts features from the audio recordings that are related to Carnatic music, which are then used to create musically meaningful relationships between all of the items in the database. The application displays the content of these items, allowing users to navigate through the collection by identifying and showing other information that is related to the currently viewed item, either by showing the relationships between them or by using culturally relevant similarity measures. The basic architecture and the design principles developed are reusable for other music collections with different characteristics., The CompMusic project is funded by the European Research Council under the European Union’s Seventh Framework Program (FP7/2007-2013) / ERC grant agreement 267583.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Characterization of the freesound online community

  • Font Corbera, Frederic
  • Roma Trepat, Gerard
  • Herrera Boyer, Perfecto, 1964-
  • Serra, Xavier
There are many online communities with membergenerated and openly available multimedia content. Their success/ndepends on having active contributing users and on producing useful content. With this criterion, the community of sound practitioners that has emerged in Freesound is a successful case of interest to be studied. But to understand it and support it further we need an appropriate analysis methodology. In this paper we propose some qualitative and quantitative approaches for its characterization, focusing on the analysis of organizational structure, shared goals, user interactions and vocabulary sharing. We think that the proposed approach can be applied to other/nonline communities with similar characteristics., This research was partly funded by the European Research Council under the European Unions Seventh Framework Program, as part of the CompMusic/nproject (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A Collection of music scores for corpus based jingju singing research

  • Caro Repetto, Rafael
  • Serra, Xavier
Comunicació presentada a: ISMIR 2017, celebrat a Suzhou, Xina, del 23 al 27 d'octubre de 2017, The MIR research on jingju (also known as Beijing or Peking opera) music has taken audio as the main source of information. Music scores are an important resource for the musicological research of this tradition, but no machine readable ones have been available for computational analysis. In order to explore the potential of symbolic score data for jingju music research, we have expanded the Comp- Music Jingju Music Corpus, which contains mostly audio, with a collection of 92 machine readable scores, for a total of 897 melodic lines. Since our purpose is the study of jingju singing in terms of its musical system elements, we have selected the arias used as examples in reference jingju music textbooks. The collection is accompanied by scores metadata, curated annotations per score and melodic line, and a set of software tools for extracting statistical information from it. All the gathered data and developed software are available for research purposes. In this paper we first discuss the culture specific concepts that are needed for understanding the contents of the collection, followed by a detailed description of it. We then present a series of computational analyses performed on the scores and discuss some musicological findings., This research is funded by the European Research Council under the European Union’s Seventh Framework Program (FP7/2007-2013), as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Computational approaches for the understanding of melody in carnatic music

  • Koduri, Gopala Krishna
  • Miron, Marius
  • Serrà Julià, Joan
  • Serra, Xavier
The classical music traditions of the Indian subcontinent,/nHindustani and Carnatic, offer an excellent ground on which/nto test the limitations of current music information research/napproaches. At the same time, studies based on these music/ntraditions can shed light on how to solve new and complex/nmusic modeling problems. Both traditions have very distinct/ncharacteristics, specially compared with western ones:/nthey have developed unique instruments, musical forms, performance/npractices, social uses and context. In this article,/nwe focus on the Carnatic music tradition of south India, especially/non its melodic characteristics. We overview the/ntheoretical aspects that are relevant for music information/nresearch and discuss the scarce computational approaches/ndeveloped so far. We put emphasis on the limitations of the/ncurrent methodologies and we present open issues that have/nnot yet been addressed and that we believe are important to/nbe worked on., The research leading to these results has received funding/nfrom the European Research Council under the European/nUnion’s Seventh Framework Programme (FP7/2007-2013)/n/ ERC grant agreement 267583 (CompMusic).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A Simple fusion of deep and shallow learning for acoustic scene classification

  • Fonseca, Eduardo
  • Gong, Rong
  • Serra, Xavier
Comunicació presentada a: 15th Sound and Music Computing Conference (SMC2018). Sonic crossing, celebrat a Limassol, Xipre, del 4 al 7 de juliol de 2018., In the past, Acoustic Scene Classification systems havebeen based on hand crafting audio features that are input toa classifier. Nowadays, the common trend is to adopt datadriven techniques, e.g., deep learning, where audio repre-sentations are learned from data. In this paper, we proposea system that consists of a simple fusion of two methods ofthe aforementioned types: a deep learning approach wherelog-scaled mel-spectrograms are input to a convolutionalneural network, and a feature engineering approach, wherea collection of hand-crafted features is input to a gradientboosting machine. We first show that both methods pro-vide complementary information to some extent. Then, weuse a simple late fusion strategy to combine both meth-ods. We report classification accuracy of each method in-dividually and the combined system on the TUT AcousticScenes 2017 dataset. The proposed fused system outper-forms each of the individual methods and attains a classifi-cation accuracy of 72.8% on the evaluation set, improvingthe baseline system by 11.8%., This work is partially supported by the European Union’sHorizon 2020 research and innovation programme undergrant agreement No 688382 “AudioCommons”, and theEuropean Research Council under the European Union’sSeventh Framework Program, as part of the CompMusicproject (ERC grant agreement 267583), and a Google Fac-ulty Research Award 2017. We are grateful for the GPUsdonated by NVidia.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Linking scores and audio recordings in makam music of Turkey

  • Sentürk, Sertan
  • Holzapfel, Andre
  • Serra, Xavier
The most relevant representations of music are notations and audio recordings, each of which emphasizes a particular perspective and promotes different approximations in the analysis and understanding of music. Linking these two representations and analysing them jointly should help to better study many musical facets by being able to combine complementary analysis methodologies. In order to develop accurate linking methods, we have to take into account the specificities of a given type of music. In this paper, we present a method for linking musically relevant sections in a score of a piece from makam music of Turkey (MMT) to the corresponding time intervals of an audio recording of the same piece. The method starts by extracting relevant features from the score and from the audio recording. The features of a given score section are compared with the features of the audio recording to find the candidate links in the audio for that score section. Next, using the sequential section information stored in the score, it selects the most likely links. The method is tested on a dataset consisting of instrumental and vocal compositions of MMT, achieving 92.1% and 96.9% -scores on the instrumental and vocal pieces, respectively. Our results show the importance of culture-specific and knowledge-based approaches in music information processing., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A Turkish makam music symbolic database for music information retrieval: SymbTr

  • Karaosmanoğlu, M. Kemal
Turkish makam music needs a comprehensive database/nfor public consumption, to be used in MIR. This article/nintroduces SymbTr, a Turkish Makam Music Symbolic/nRepresentation Database, aimed at filling this void./nSymbTr consists of musical information in text, PDF, and/nMIDI formats. Raw data, drawn from reliable sources,/nand consisting of 1,700 musical pieces in Turkish art and/nfolk music was processed featuring distinct examples in/n155 diverse makams, 100 usuls and 48 forms. Special/ncare was devoted to selection of works that scatter across/na broad historical time span and were among those still/nperformed today. Total number of musical notes in these/npieces was 630,000, corresponding to a nominal playback/ntime of 72 hours. Synthesized sounds particular to Turkish/nmakam music were used in MIDI playback, and transcription//nplayback errors were corrected by input from/nexperts. Symbolic representation data, open to the public,/nis output from a computer program developed exclusively/nfor Turkish makam music. SymbTr was designed as a/nwholesome representation of aforementioned distinct auditory/nand visual features that distinguish Turkish makam/nmusic from other music genres. This article explains the/ndatabase format in detail, and also provides, through examples,/nstatistical information on pitch/interval allocation/nand distribution., This research was partly funded by the European Research/nCouncil under the European Union's Seventh/nFramework Program, as part of the CompMusic project/n(ERC grant agreement 267583).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Singing voice separation by harmonic modeling

  • Dzhambazov, Georgi Bogomilov
  • Serra, Xavier
Comunicació presentada a: 17th International Society for Music Information Retrieval Conference (ISMIR) celebrat del 7 a l'11 d'agost de 2016 a Nova York, Estats Units d'Amèrica., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583) and partly by the AGAUR research grant.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Incorporating features of distribution and progression for automatic Makam classification

  • Ünal, Erdem
  • Bozkurt, Baris
  • Karaosmanoğlu, M. Kemal
Comunicació presentada al 2nd CompMusic Workshop, celebrat els dies 12 i 13 de juliol de 2012 a Istanbul (Turquia), organitzat per CompMusic., Automatic classification of makams from symbolic data is a rarely studied topic. In this paper, first a review of an n-gram based approach is presented using various representations of the symbolic data. While a high degree of precision can be obtained, confusion happens mainly for makams using (almost) the same scale and pitch hierarchy but differ in overall melodic progression, seyir. To further improve the system, first n-gram based classification is tested for various sections of the piece to take into account a feature of the seyir that melodic progression starts in a certain region of the scale. In a second test, a hierarchical classification structure is designed which uses n-grams and seyir features in different levels to further improve the system., This work was funded in part by the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013) / ERC grant agreement 267583 (CompMusic) and in part by TÜBİTAK ARDEB grant no:3501-109E196. All staff notation representations used in this paper is taken from Mus2okur software, a digital encyclopedia for Turkish music (http://www.musiki.org/).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Automatic alignment of long syllables in a cappella Beijing opera

  • Dzhambazov, Georgi Bogomilov
  • Yang, Yile
  • Caro Repetto, Rafael
  • Serra, Xavier
Comunicació presentada al 6th International Workshop on Folk Music Analysis, celebrat els dies 15 a 17 de juny de 2016 a Dublín, Irlanda., In this study we propose how to modify a standard approach for text-to-speech alignment to apply in the case of alignment of lyrics and singing voice. We model phoneme durations by means of a duration-explicit hidden Markov model (DHMM) phonetic recognizer based on MFCCs. The phoneme durations are empirically set in a probabilistic way, based on prior knowledge about the lyrics structure and metric principles, specific for the Beijing opera music tradition. Phoneme models are GMMs trained directly on a small corpus of annotated singing voice. The alignment is evaluated on a cappella material from Beijing opera, which is characterized by its particularly long syllable durations. Results show that the incorporation of music-specific knowledge results in a very high alignment accuracy, outperforming significantly a baseline HMM-based approach., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583) and partly by the AGAUR research grant.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A method for structural analysis of Ottoman-Turkish Makam music scores

  • Sentürk, Sertan
  • Serra, Xavier
Comunicació presentada al 6th International Workshop on Folk Music Analysis, celebrat els dies 15 a 17 de juny de 2016 a Dublín, Irlanda.Comunicació presentada al 6th International Workshop on Folk Music Analysis, celebrat els dies 15 a 17 de juny de 2016 a Dublín, Irlanda., From a computational perspective, structural analysis of OttomanTurkish makam music (OTMM) is a research topic that has not been addressed thoroughly. In this paper we propose a method, which processes machine-readable music scores of OTMM to extract and semiotically describe the melodic and lyrical organization of the music piece automatically using basic string similarity and graph analysis techniques. The proposed method is used to identify around 50000 phrases in 1300 music scores and 21500 sections in 1770 scores, respectively. The obtained information may be useful for relevant research in music education and musicology, and it has already been used to aid several computational tasks such as music score content validation, digital music engraving and audio-score alignment., This work is partly supported by the European Research Council under the European Union’s Seventh Framework Program, as part of the CompMusic project (ERC grant agreement 267583).