Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 30
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

On the interrelation between listener characteristics and the perception of emotion in classical orchestra music

  • Schedl, Markus
  • Gómez Gutiérrez, Emilia, 1975-
  • Trent, Erika S.
  • Tkalčič, Marko
  • Eghbal-Zadeh, Hamid
  • Martorell Domínguez, Agustín
This study deals with the strong relationship between emotions and music, investigating three main research questions: (RQ1) Are there differences in human music perception (e.g., emotions, tempo, instrumentation, and complexity), according to musical education, experience, demographics, and personality traits?; (RQ2) Do certain perceived music characteristics correlate (e.g., tension and sadness), irrespective of a particular listener's background or personality?; (RQ3) Does human perception of music characteristics, such as emotions and tempo, correlate with descriptors extracted from music audio signals? To investigate our research questions, we conducted two user studies focusing on different groups of subjects. We used web-based surveys to collect information about demographics, listening habits, musical education, personality, and perceptual ratings with respect to perceived emotions, tempo, complexity, and instrumentation for 15 segments of Beethoven's 3rd symphony, “Eroica”. Our experiments showed that all three research questions can be affirmed, at least partly. We found strong support for RQ2 and RQ3, while RQ1 could be confirmed only for some perceptual aspects and user groups., This research is supported by the EU-FP7 project no. 601166 (“PHENICX”), the Austrian Science Fund (FWF): P25655 and the Spanish Government (CASAS project TIN2015-70816-R).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A personality-based adaptive system for visualizing classical music performances

  • Schedl, Markus
  • Melenhorst, Mark
  • Liem, Cynthia C. S.
  • Martorell Domínguez, Agustín
  • Mayor, Oscar
  • Tkalčič, Marko
To enhance the experience of listening to classical orchestra music, either in the concert hall or at home, we present a personalized system that integrates three visualization/interaction concepts: Score Follower (points to the current position in the score), Orchestra Layout (illustrates instruments that are currently playing and their dynamics), and Structure Visualization (visualizes structural elements such as themes or motifs). Motivated by previous literature that found evidence for connections between personality and music consumption and preference, we first assessed in a user study to which extent personality traits and music visualization preferences correlate. Measuring preference via pragmatic quality and personality traits according to the Big Five Inventory (BFI) questionnaire, we found substantial interconnections between them. These translate into rules relating certain personality traits (e.g., extraversion or agreeableness) to preference rankings of the visualizations. In the proposed personality-based system, users are grouped into four clusters according to their answers to the most significant personality questions determined in the study. The order of the visualizations for a given user is adapted with respect to the ranking preferred by other users in the same cluster. Evaluation of the system was carried out by a second user study that showed a significantly higher normalized discounted cumulative gain (NDCG) for the personalized system in comparison to a system with randomized order of the visualizations., This work is supported by the EU FP7-ICT-2011-9 project 601166 (“PHENICX”) and the Austrian Science Fund (FWF): P25655.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

An analysis of agreement in classical music perception and its relationship to listener characteristics

  • Schedl, Markus
  • Eghbal-Zadeh, Hamid
  • Gómez Gutiérrez, Emilia, 1975-
  • Tkalčič, Marko
Comunicació presentada a la ISMR 2016, 17th International Society for Music Information Retrieval Conference, celebrada els dies 7 a 11 d'agost de 2016 a New York City., We present a study, carried out on 241 participants, which investigates on classical music material the agreement of listeners on perceptual music aspects (related to emotion, tempo, complexity, and instrumentation) and the relationship between listener characteristics and these aspects. For the currently popular task of music emotion recognition, the former question is particularly important when defining a ground truth of emotions perceived in a given music collection. We characterize listeners via a range of factors, including demographics, musical inclination, experience, and education, and personality traits. Participants rate the music material under investigation, i.e., 15 expert-defined segments of Beethoven’s 3rd symphony, “Eroica”, in terms of 10 emotions, perceived tempo, complexity, and number of instrument groups. Our study indicates only slight agreement on most perceptual aspects, but significant correlations between several listener characteristics and perceptual qualities., This research is supported by the Austrian Science Fund (FWF): P25655 and was made possible by the EU FP7 project no. 601166 (“PHENICX”).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Evaluation and combination of pitch estimation methods for melody extraction in symphonic classical music

  • Bosch, Juan J.
  • Marxer Piñón, Ricard
  • Gómez Gutiérrez, Emilia, 1975-
The extraction of pitch information is arguably one of the most important tasks in automatic music description systems. However, previous research and evaluation datasets dealing with pitch estimation focused on relatively limited kinds of musical data. This work aims to broaden this scope by addressing symphonic western classical music recordings, focusing on pitch estimation for melody extraction. This material is characterized by a high number of overlapping sources, and by the fact that the melody may be played by different instrumental sections, often alternating within an excerpt. We evaluate the performance of eleven state-of-the-art pitch salience functions, multipitch estimation and melody extraction algorithms when determining the sequence of pitches corresponding to the main melody in a varied set of pieces. An important contribution of the present study is the proposed evaluation framework, including the annotation methodology, generated dataset and evaluation metrics. The results show that the assumptions made by certain methods hold better than others when dealing with this type of music signal, leading to a better performance. Additionally, we propose a simple method for combining the output of several algorithms, with promising results., This work is supported by the European Union Seventh Framework Programme FP7 / 2007-2013 through the PHENICX project (grant agreement no. 601166). This work is also (partly) supported by the Spanish Ministry of Economy and Competitiveness/nunder the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Towards concurrent multi-tasking in shareable interfaces

  • Fernandez Julià, Carles, 1984-
  • Jordà Puig, Sergi
Shareable interfaces, those that can be interacted simultaneously by several users, are a common tool used both in CSCW research and in real world applications. They tend however to lack a capability that has been traditionally relevant to the usefulness of computing systems: multi-tasking. In this paper we explain why a combination of the multi-user features of shareable interfaces and the multi-tasking capabilities of general-purpose computing, could be relevant for building useful systems, and why these features are not present today in most of the current prototypes and systems. We also discuss possible approaches for solving the problems that prevent shareable interfaces to fully support multi-tasking, and we present a novel approach based on a distributed, application-centered, content-based gesture disambiguation. We describe how an already existing framework, GestureAgents, implements this new approach, focusing on expanding the description of the relevant elements related to this problem, and conclude with some example applications and a discussion., The research leading to these results has received funding from the European Union Seventh Framework Programme FP7 / 2007–2013 through PHENICX project under grant agreement n° 601166.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Music information retrieval: recent developments and applications

  • Schedl, Markus
  • Gómez Gutiérrez, Emilia, 1975-
  • Urbano, Julián
We provide a survey of the field of Music Information Retrieval (MIR), in particular paying attention to latest developments, such as semantic auto-tagging and user-centric retrieval and recommendation approaches. We first elaborate on well-established and proven methods for feature extraction and music indexing, from both the audio signal and contextual data sources about music items, such as web pages or collaborative tags. These in turn enable a wide variety of music retrieval tasks, such as semantic music search or music identification (“query by example”). Subsequently, we review current work on user analysis and modeling in the context of music recommendation and retrieval, addressing the recent trend towards user-centric and adaptive approaches and systems. A discussion follows about the important aspect of how various MIR approaches to different problems are evaluated and compared. Eventually, a discussion about the major open challenges concludes the survey., This work is supported by the Austrian Science Fund (FWF): P22856 and P25655, as well as by the European Union Seventh Framework Programme FP7 / 2007-2013 through the PHENICX project under grant agreement no. 601166, the A4U postdoctoral grants programme, and the Junta of Andalusia through the COFLA2 project (P12-TIC-1362). The authors would further like to thank Masataka Goto, Mohamed Sordo, and Edith Law for granting permission to include their visual material in this survey, and Juan J. Bosch for his comments on the manuscript. Furthermore, the authors would like to express their gratitude to the anonymous reviewers for their highly valuable suggestions for improving the manuscript.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Enriched multimodal representations of music performances: online access and visualization

  • Maestre Gómez, Esteban
  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Llimona Torras, Quim
  • Mayor, Oscar
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Wanderley, Marcelo
The authors provide a first-person outlook on the technical challenges involved in the recording, analysis, archiving, and cloud-based interchange of multimodal string quartet performance data as part of a collaborative research project on ensemble music making. To facilitate the sharing of their own collection of multimodal recordings and extracted descriptors and annotations, they developed a hosting platform through which multimodal data (audio, video, motion capture, and derived signals) can be stored, visualized, annotated, and selectively retrieved via a web interface and a dedicated API. This article offers a twofold contribution: the authors open their collection of enriched multimodal recordings, the Quartet dataset, to the community, and they introduce and enable access to their multimodal data exchange platform and web application, the Repovizz system. This article is part of a special issue on multimedia technologies for enriched music., This work was partially funded by the EU FP7 project numbers 250026 (SIEMPRE) and 601166 (PHENICX), the EU Marie Curie Integration Grant PIOF-GA-2012-331270, and the AGAUR Catalan Public Funding Agency. Moreover, the authors thank the BRAMS Laboratory and the contributions by Erika Donald, Marcello Giordano, Carolina B. Medeiros, Marina Azón, Guillermo Navarro, and Jordi Llop.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

A comparison of melody extraction methods based on source-filter modelling

  • Bosch, Juan J.
  • Bittner, Rachel M.
  • Salamon, Justin
  • Gómez Gutiérrez, Emilia, 1975-
This work explores the use of source-filter models for pitch/nsalience estimation and their combination with different/npitch tracking and voicing estimation methods for automatic/nmelody extraction. Source-filter models are used/nto create a mid-level representation of pitch that implicitly/nincorporates timbre information. The spectrogram of/na musical audio signal is modelled as the sum of the leading/nvoice (produced by human voice or pitched musical instruments)/nand accompaniment. The leading voice is then/nmodelled with a Smoothed Instantaneous Mixture Model/n(SIMM) based on a source-filter model. The main advantage/nof such a pitch salience function is that it enhances/nthe leading voice even without explicitly separating it from/nthe rest of the signal. We show that this is beneficial/nfor melody extraction, increasing pitch estimation accuracy/nand reducing octave errors in comparison with simpler/npitch salience functions. The adequate combination with/nvoicing detection techniques based on pitch contour characterisation/nleads to significant improvements over stateof-the-art/nmethods, for both vocal and instrumental music., This work is partially supported by the European Union/nunder the PHENICX project (FP7-ICT-601166) and the/nSpanish Ministry of Economy and Competitiveness under/nCASAS project (TIN2015-70816-R) and Maria de Maeztu/nUnits of Excellence Programme (MDM-2015-0502)
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

What is the effect of audio quality on the robustness of MFCCs and chroma features?

  • Urbano, Julián
  • Bogdanov, Dmitry
  • Herrera Boyer, Perfecto, 1964-
  • Gómez Gutiérrez, Emilia, 1975-
  • Serra, Xavier
Music Information Retrieval is largely based on descriptors computed from audio signals, and in many practical applications they are to be computed on music corpora containing audio files encoded in a variety of lossy formats. Such encodings distort the original signal and therefore may affect the computation of descriptors. This raises the question of the robustness of these descriptors across various audio encodings. We examine this assumption for the case of MFCCs and chroma features. In particular, we analyze their robustness to sampling rate, codec, bitrate, frame size and music genre. Using two different audio analysis tools over a diverse collection of music tracks, we compute several statistics to quantify the robustness of the resulting descriptors, and then estimate the practical effects for a sample task like genre classification., This work is partially supported by an A4U postdoctoral grant and projects SIGMUS (TIN2012-36650), Comp- Music (ERC 267583), PHENICX (ICT-2011.8.2) and GiantSteps (ICT-2013-10).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Mapping by observation: building a user-tailored conducting system from spontaneous movements

  • Sarasúa Berodia, Álvaro
  • Urbano, Julián
  • Gómez Gutiérrez, Emilia, 1975-
Metaphors are commonly used in interface design within Human-Computer Interaction (HCI). Interface metaphors provide users with a way to interact with the computer that resembles a known activity, giving instantaneous knowledge or intuition about how the interaction works. A widely used one in Digital Musical Instruments (DMIs) is the conductor-orchestra metaphor, where the orchestra is considered as an instrument controlled by the movements of the conductor. We propose a DMI based on the conductor metaphor that allows to control tempo and dynamics and adapts its mapping specifically for each user by observing spontaneous conducting movements (i.e., movements performed on top of fixed music without any instructions). We refer to this as mapping by observation given that, even though the system is trained specifically for each user, this training is not done explicitly and consciously by the user. More specifically, the system adapts its mapping based on the tendency of the user to anticipate or fall behind the beat and observing the Motion Capture descriptors that best correlate to loudness during spontaneous conducting. We evaluate the proposed system in an experiment with twenty four (24) participants where we compare it with a baseline that does not perform this user-specific adaptation. The comparison is done in a context where the user does not receive instructions and, instead, is allowed to discover by playing. We evaluate objective and subjective measures from tasks where participants have to make the orchestra play at different loudness levels or in synchrony with a metronome. Results of the experiment prove that the usability of the system that automatically learns its mapping from spontaneous movements is better both in terms of providing a more intuitive control over loudness and a more precise control over beat timing. Interestingly, the results also show a strong correlation between measures taken from the data used for training and the improvement introduced by the adapting system. This indicates that it is possible to estimate in advance how useful the observation of spontaneous movements is to build user-specific adaptations. This opens interesting directions for creating more intuitive and expressive DMIs, particularly in public installations., This work was supported by the European Union Seventh Framework Programme FP7 / 2007-2013 through the PHENICX project (grant agreement no. 601166) and by the CASAS project (TIN2015-70816-R).
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Melody extraction based on a source-filter model using pitch contour selection

  • Bosch, Juan J.
  • Gómez Gutiérrez, Emilia, 1975-
This work proposes a melody extraction method which combines a pitch salience function based on source-filter modelling with melody tracking based on pitch contour selection. We model the spectrogram of a musical audio signal as the sum of the leading voice and accompaniment. The leading voice is modelled with a Smoothed Instantaneous Mixture Model (SIMM), and the accompaniment is modelled with a Non-negative Matrix Factorization (NMF). The main benefit of this representation is that it incorporates timbre information, and that the leading voice is enhanced, even without an explicit separation from the rest of the signal. Two different salience functions based on SIMM are proposed, in order to adapt the output of such model to the pitch contour based tracking. Candidate melody pitch contours are then created by grouping pitch sequences, using auditory streaming cues. Finally, melody pitch contours are selected using a set of heuristic rules based on contour characteristics and smoothness constraints. The evaluation on a large set of challenging polyphonic music material, shows that the proposed salience functions help increasing the salience of melody pitches in comparison to similar methods. The complete melody extraction methods also achieve a higher overall accuracy than state-of-the-art approaches when evaluated on both vocal and instrumental music., This work is partially supported by the European Union under the PHENICX project (FP7-ICT-601166) and the Spanish Ministry of Economy and Competitiveness under CASAS project (TIN2015-70816-R) and Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Hierarchical multi-scale set-class analysis

  • Martorell Domínguez, Agustín
  • Gómez Gutiérrez, Emilia, 1975-
This work presents a systematic methodology for set-class surface analysis using temporal multi-scale techniques. The method extracts the set-class content of all the possible temporal segments, addressing the representational problems derived from the massive overlapping of segments. A time versus time-scale representation, named class-scape, provides a global hierarchical overview of the class content in the piece, and it serves as a visual index for interactive inspection. Additional data structures summarize the set-class inclusion relations over time and quantify the class and subclass content in pieces or collections, helping to decide about sets of analytical interest. Case studies include the comparative subclass characterization of diatonicism in Victoria's masses (in Ionian mode) and Bach's preludes and fugues (in major mode), as well as the structural analysis of Webern's Variations for piano op. 27, under different class-equivalences., This work was supported by the EU Seventh Framework Programme FP7/2007-2013 through PHENICX project [grant no. 601166].
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Becoming the maestro - A game to enhance curiosity for classical music

  • Sarasúa Berodia, Álvaro
  • Melenhorst, Mark
  • Fernandez Julià, Carles, 1984-
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a la 2016 8th International Conference on Games and Virtual Worlds for Serious Applications (VS-Games), celebrada els dies 7 a 9 de setembre de 2016 a Barcelona, Espanya., Classical music is associated with a set of social and cognitive barriers that prevent potentially interested novice classical music listeners from attending concerts. The possible strategies to overcome these barriers and make the genre appealing to new audiences are diverse. In this paper, we present Becoming the Maestro, a game where users impersonate the conductor of an orchestra during a concert in a way that allows them to better understand musical concepts of symphonic music. A user study of the game prototype with 20 subjects proved that the game succeeded in increasing curiosity for classical music., This research received funding from the European Union Seventh Framework Programme FP7 / 2007-2013 PHENICX project (grant agreement no. 601166).
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Multi-feature beat tracking

  • Zapata González, José Ricardo
  • Davies, Matthew E. P.
  • Gómez Gutiérrez, Emilia, 1975-
A recent trend in the field of beat tracking for musical audio signals has been to explore techniques for measuring the level of agreement and disagreement between a committee of beat tracking algorithms. By using beat tracking evaluation methods to compare all pairwise combinations of beat tracker outputs, it has been shown that selecting the beat tracker which most agrees with the remainder of the committee, on a song-by-song basis, leads to improved performance which surpasses the accuracy of any individual beat tracker used on its own. In this paper we extend this idea towards presenting a single, standalone beat tracking solution which can exploit the benefit of mutual agreement without the need to run multiple separate beat tracking algorithms. In contrast to existing work, we re-cast the problem as one of selecting between the beat outputs resulting from a single beat tracking model with multiple, diverse input features. Through extended evaluation on a large annotated database, we show that our multi-feature beat tracker can outperform the state of the art, and thereby demonstrate that there is sufficient diversity in input features for beat tracking, without the need for multiple tracking models., This work was supported in part by the R+I+D Ph.D. scholarship of Universidad Pontificia Bolivariana and Colciencias (Colombia), the projects of the Spanish Ministry of Science and Innovation DRIMS (MICINN-TIN2009-14247-C02-01), SIGMUS (MINECO-TIN2012-36650) and Mires (EC-7PM-MIReS), and in part by the Media Arts and Technologies project (MAT), NORTE-07-0124-FEDER-000061, financed by the North Portugal Regional Operational Programme (ON.2 O Novo Norte), under the National Strategic Reference Framework (NSRF), through the European Regional Development Fund (ERDF) by national funds, through the Portuguese funding agency, Fundação para a Ciência e a Tecnologia (FCT), the FCT post-doctoral grant (SFRH/BPD/88722/2012), the PHENICX (EU: FP7 2007/2013 grant agreement n 601166) and COFLA (Junta de Andaluca P09-TIC-4840) projects.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Improving score-informed source separation for classical music through note refinement

  • Miron, Marius
  • Carabias Orti, Julio J.
  • Janer Mestres, Jordi
Comunicació presentada a la 16th International Society for Music Information Retrieval Conference (ISMIR 2015), celebrada els dies 26 a 30 d'octubre de 2015 a Màlaga, Espanya., Signal decomposition methods such as Non-negative Matrix Factorization (NMF) demonstrated to be a suitable approach for music signal processing applications, including sound source separation. To better control this decomposition, NMF has been extended using prior knowledge and parametric models. In fact, using score information considerably improved separation results. Nevertheless, one of the main problems of using score information is the misalignment between the score and the actual performance. A potential solution to this problem is the use of audio to score alignment systems. However, most of them rely on a tolerance window that clearly affects the separation results. To overcome this problem, we propose a novel method to refine the aligned score at note level by detecting both, onset and offset for each note present in the score. Note refinement is achieved by detecting shapes and contours in the estimated instrument-wise time activation (gains) matrix. Decomposition is performed in a supervised way, using training instrument models and coarsely-aligned score information. The detected contours define time-frequency note boundaries, and they increase the sparsity. Finally, we have evaluated our method for informed source separation using a dataset of Bach chorales obtaining satisfactory results, especially in terms of SIR., This work was supported by the European Commission, FP7 (Seventh Framework Programme), STREP project, ICT- 2011.8.2 ICT for access to cultural resources, grant agreement No 601166. Phenicx Project
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

An analysis of agreement in classical music perception and its relationship to listener characteristics

  • Schedl, Markus
  • Eghbal-Zadeh, Hamid
  • Gómez Gutiérrez, Emilia, 1975-
  • Tkalčič, Marko
Comunicació presentada a la ISMR 2016, 17th International Society for Music Information Retrieval Conference, celebrada els dies 7 a 11 d'agost de 2016 a New York City., We present a study, carried out on 241 participants, which investigates on classical music material the agreement of listeners on perceptual music aspects (related to emotion, tempo, complexity, and instrumentation) and the relationship between listener characteristics and these aspects. For the currently popular task of music emotion recognition, the former question is particularly important when defining a ground truth of emotions perceived in a given music collection. We characterize listeners via a range of factors, including demographics, musical inclination, experience, and education, and personality traits. Participants rate the music material under investigation, i.e., 15 expert-defined segments of Beethoven’s 3rd symphony, “Eroica”, in terms of 10 emotions, perceived tempo, complexity, and number of instrument groups. Our study indicates only slight agreement on most perceptual aspects, but significant correlations between several listener characteristics and perceptual qualities., This research is supported by the Austrian Science Fund (FWF): P25655 and was made possible by the EU FP7 project no. 601166 (“PHENICX”).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A comparison of melody extraction methods based on source-filter modelling

  • Bosch, Juan J.
  • Bittner, Rachel M.
  • Salamon, Justin
  • Gómez Gutiérrez, Emilia, 1975-
This work explores the use of source-filter models for pitch/nsalience estimation and their combination with different/npitch tracking and voicing estimation methods for automatic/nmelody extraction. Source-filter models are used/nto create a mid-level representation of pitch that implicitly/nincorporates timbre information. The spectrogram of/na musical audio signal is modelled as the sum of the leading/nvoice (produced by human voice or pitched musical instruments)/nand accompaniment. The leading voice is then/nmodelled with a Smoothed Instantaneous Mixture Model/n(SIMM) based on a source-filter model. The main advantage/nof such a pitch salience function is that it enhances/nthe leading voice even without explicitly separating it from/nthe rest of the signal. We show that this is beneficial/nfor melody extraction, increasing pitch estimation accuracy/nand reducing octave errors in comparison with simpler/npitch salience functions. The adequate combination with/nvoicing detection techniques based on pitch contour characterisation/nleads to significant improvements over stateof-the-art/nmethods, for both vocal and instrumental music., This work is partially supported by the European Union/nunder the PHENICX project (FP7-ICT-601166) and the/nSpanish Ministry of Economy and Competitiveness under/nCASAS project (TIN2015-70816-R) and Maria de Maeztu/nUnits of Excellence Programme (MDM-2015-0502)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Hierarchical multi-scale set-class analysis

  • Martorell Domínguez, Agustín
  • Gómez Gutiérrez, Emilia, 1975-
This work presents a systematic methodology for set-class surface analysis using temporal multi-scale techniques. The method extracts the set-class content of all the possible temporal segments, addressing the representational problems derived from the massive overlapping of segments. A time versus time-scale representation, named class-scape, provides a global hierarchical overview of the class content in the piece, and it serves as a visual index for interactive inspection. Additional data structures summarize the set-class inclusion relations over time and quantify the class and subclass content in pieces or collections, helping to decide about sets of analytical interest. Case studies include the comparative subclass characterization of diatonicism in Victoria's masses (in Ionian mode) and Bach's preludes and fugues (in major mode), as well as the structural analysis of Webern's Variations for piano op. 27, under different class-equivalences., This work was supported by the EU Seventh Framework Programme FP7/2007-2013 through PHENICX project [grant no. 601166].
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Becoming the maestro - A game to enhance curiosity for classical music

  • Sarasúa Berodia, Álvaro
  • Melenhorst, Mark
  • Fernandez Julià, Carles, 1984-
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a la 2016 8th International Conference on Games and Virtual Worlds for Serious Applications (VS-Games), celebrada els dies 7 a 9 de setembre de 2016 a Barcelona, Espanya., Classical music is associated with a set of social and cognitive barriers that prevent potentially interested novice classical music listeners from attending concerts. The possible strategies to overcome these barriers and make the genre appealing to new audiences are diverse. In this paper, we present Becoming the Maestro, a game where users impersonate the conductor of an orchestra during a concert in a way that allows them to better understand musical concepts of symphonic music. A user study of the game prototype with 20 subjects proved that the game succeeded in increasing curiosity for classical music., This research received funding from the European Union Seventh Framework Programme FP7 / 2007-2013 PHENICX project (grant agreement no. 601166).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Improving score-informed source separation for classical music through note refinement

  • Miron, Marius
  • Carabias Orti, Julio J.
  • Janer Mestres, Jordi
Comunicació presentada a la 16th International Society for Music Information Retrieval Conference (ISMIR 2015), celebrada els dies 26 a 30 d'octubre de 2015 a Màlaga, Espanya., Signal decomposition methods such as Non-negative Matrix Factorization (NMF) demonstrated to be a suitable approach for music signal processing applications, including sound source separation. To better control this decomposition, NMF has been extended using prior knowledge and parametric models. In fact, using score information considerably improved separation results. Nevertheless, one of the main problems of using score information is the misalignment between the score and the actual performance. A potential solution to this problem is the use of audio to score alignment systems. However, most of them rely on a tolerance window that clearly affects the separation results. To overcome this problem, we propose a novel method to refine the aligned score at note level by detecting both, onset and offset for each note present in the score. Note refinement is achieved by detecting shapes and contours in the estimated instrument-wise time activation (gains) matrix. Decomposition is performed in a supervised way, using training instrument models and coarsely-aligned score information. The detected contours define time-frequency note boundaries, and they increase the sparsity. Finally, we have evaluated our method for informed source separation using a dataset of Bach chorales obtaining satisfactory results, especially in terms of SIR., This work was supported by the European Commission, FP7 (Seventh Framework Programme), STREP project, ICT- 2011.8.2 ICT for access to cultural resources, grant agreement No 601166. Phenicx Project
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Enriched multimodal representations of music performances: online access and visualization

  • Maestre Gómez, Esteban
  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Llimona Torras, Quim
  • Mayor, Oscar
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Wanderley, Marcelo
The authors provide a first-person outlook on the technical challenges involved in the recording, analysis, archiving, and cloud-based interchange of multimodal string quartet performance data as part of a collaborative research project on ensemble music making. To facilitate the sharing of their own collection of multimodal recordings and extracted descriptors and annotations, they developed a hosting platform through which multimodal data (audio, video, motion capture, and derived signals) can be stored, visualized, annotated, and selectively retrieved via a web interface and a dedicated API. This article offers a twofold contribution: the authors open their collection of enriched multimodal recordings, the Quartet dataset, to the community, and they introduce and enable access to their multimodal data exchange platform and web application, the Repovizz system. This article is part of a special issue on multimedia technologies for enriched music., This work was partially funded by the EU FP7 project numbers 250026 (SIEMPRE) and 601166 (PHENICX), the EU Marie Curie Integration Grant PIOF-GA-2012-331270, and the AGAUR Catalan Public Funding Agency. Moreover, the authors thank the BRAMS Laboratory and the contributions by Erika Donald, Marcello Giordano, Carolina B. Medeiros, Marina Azón, Guillermo Navarro, and Jordi Llop.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Melody extraction based on a source-filter model using pitch contour selection

  • Bosch, Juan J.
  • Gómez Gutiérrez, Emilia, 1975-
This work proposes a melody extraction method which combines a pitch salience function based on source-filter modelling with melody tracking based on pitch contour selection. We model the spectrogram of a musical audio signal as the sum of the leading voice and accompaniment. The leading voice is modelled with a Smoothed Instantaneous Mixture Model (SIMM), and the accompaniment is modelled with a Non-negative Matrix Factorization (NMF). The main benefit of this representation is that it incorporates timbre information, and that the leading voice is enhanced, even without an explicit separation from the rest of the signal. Two different salience functions based on SIMM are proposed, in order to adapt the output of such model to the pitch contour based tracking. Candidate melody pitch contours are then created by grouping pitch sequences, using auditory streaming cues. Finally, melody pitch contours are selected using a set of heuristic rules based on contour characteristics and smoothness constraints. The evaluation on a large set of challenging polyphonic music material, shows that the proposed salience functions help increasing the salience of melody pitches in comparison to similar methods. The complete melody extraction methods also achieve a higher overall accuracy than state-of-the-art approaches when evaluated on both vocal and instrumental music., This work is partially supported by the European Union under the PHENICX project (FP7-ICT-601166) and the Spanish Ministry of Economy and Competitiveness under CASAS project (TIN2015-70816-R) and Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Multi-feature beat tracking

  • Zapata González, José Ricardo
  • Davies, Matthew E. P.
  • Gómez Gutiérrez, Emilia, 1975-
A recent trend in the field of beat tracking for musical audio signals has been to explore techniques for measuring the level of agreement and disagreement between a committee of beat tracking algorithms. By using beat tracking evaluation methods to compare all pairwise combinations of beat tracker outputs, it has been shown that selecting the beat tracker which most agrees with the remainder of the committee, on a song-by-song basis, leads to improved performance which surpasses the accuracy of any individual beat tracker used on its own. In this paper we extend this idea towards presenting a single, standalone beat tracking solution which can exploit the benefit of mutual agreement without the need to run multiple separate beat tracking algorithms. In contrast to existing work, we re-cast the problem as one of selecting between the beat outputs resulting from a single beat tracking model with multiple, diverse input features. Through extended evaluation on a large annotated database, we show that our multi-feature beat tracker can outperform the state of the art, and thereby demonstrate that there is sufficient diversity in input features for beat tracking, without the need for multiple tracking models., This work was supported in part by the R+I+D Ph.D. scholarship of Universidad Pontificia Bolivariana and Colciencias (Colombia), the projects of the Spanish Ministry of Science and Innovation DRIMS (MICINN-TIN2009-14247-C02-01), SIGMUS (MINECO-TIN2012-36650) and Mires (EC-7PM-MIReS), and in part by the Media Arts and Technologies project (MAT), NORTE-07-0124-FEDER-000061, financed by the North Portugal Regional Operational Programme (ON.2 O Novo Norte), under the National Strategic Reference Framework (NSRF), through the European Regional Development Fund (ERDF) by national funds, through the Portuguese funding agency, Fundação para a Ciência e a Tecnologia (FCT), the FCT post-doctoral grant (SFRH/BPD/88722/2012), the PHENICX (EU: FP7 2007/2013 grant agreement n 601166) and COFLA (Junta de Andaluca P09-TIC-4840) projects.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

What is the effect of audio quality on the robustness of MFCCs and chroma features?

  • Urbano, Julián
  • Bogdanov, Dmitry
  • Herrera Boyer, Perfecto, 1964-
  • Gómez Gutiérrez, Emilia, 1975-
  • Serra, Xavier
Music Information Retrieval is largely based on descriptors computed from audio signals, and in many practical applications they are to be computed on music corpora containing audio files encoded in a variety of lossy formats. Such encodings distort the original signal and therefore may affect the computation of descriptors. This raises the question of the robustness of these descriptors across various audio encodings. We examine this assumption for the case of MFCCs and chroma features. In particular, we analyze their robustness to sampling rate, codec, bitrate, frame size and music genre. Using two different audio analysis tools over a diverse collection of music tracks, we compute several statistics to quantify the robustness of the resulting descriptors, and then estimate the practical effects for a sample task like genre classification., This work is partially supported by an A4U postdoctoral grant and projects SIGMUS (TIN2012-36650), Comp- Music (ERC 267583), PHENICX (ICT-2011.8.2) and GiantSteps (ICT-2013-10).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Mapping by observation: building a user-tailored conducting system from spontaneous movements

  • Sarasúa Berodia, Álvaro
  • Urbano, Julián
  • Gómez Gutiérrez, Emilia, 1975-
Metaphors are commonly used in interface design within Human-Computer Interaction (HCI). Interface metaphors provide users with a way to interact with the computer that resembles a known activity, giving instantaneous knowledge or intuition about how the interaction works. A widely used one in Digital Musical Instruments (DMIs) is the conductor-orchestra metaphor, where the orchestra is considered as an instrument controlled by the movements of the conductor. We propose a DMI based on the conductor metaphor that allows to control tempo and dynamics and adapts its mapping specifically for each user by observing spontaneous conducting movements (i.e., movements performed on top of fixed music without any instructions). We refer to this as mapping by observation given that, even though the system is trained specifically for each user, this training is not done explicitly and consciously by the user. More specifically, the system adapts its mapping based on the tendency of the user to anticipate or fall behind the beat and observing the Motion Capture descriptors that best correlate to loudness during spontaneous conducting. We evaluate the proposed system in an experiment with twenty four (24) participants where we compare it with a baseline that does not perform this user-specific adaptation. The comparison is done in a context where the user does not receive instructions and, instead, is allowed to discover by playing. We evaluate objective and subjective measures from tasks where participants have to make the orchestra play at different loudness levels or in synchrony with a metronome. Results of the experiment prove that the usability of the system that automatically learns its mapping from spontaneous movements is better both in terms of providing a more intuitive control over loudness and a more precise control over beat timing. Interestingly, the results also show a strong correlation between measures taken from the data used for training and the improvement introduced by the adapting system. This indicates that it is possible to estimate in advance how useful the observation of spontaneous movements is to build user-specific adaptations. This opens interesting directions for creating more intuitive and expressive DMIs, particularly in public installations., This work was supported by the European Union Seventh Framework Programme FP7 / 2007-2013 through the PHENICX project (grant agreement no. 601166) and by the CASAS project (TIN2015-70816-R).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Towards concurrent multi-tasking in shareable interfaces

  • Fernandez Julià, Carles, 1984-
  • Jordà Puig, Sergi
Shareable interfaces, those that can be interacted simultaneously by several users, are a common tool used both in CSCW research and in real world applications. They tend however to lack a capability that has been traditionally relevant to the usefulness of computing systems: multi-tasking. In this paper we explain why a combination of the multi-user features of shareable interfaces and the multi-tasking capabilities of general-purpose computing, could be relevant for building useful systems, and why these features are not present today in most of the current prototypes and systems. We also discuss possible approaches for solving the problems that prevent shareable interfaces to fully support multi-tasking, and we present a novel approach based on a distributed, application-centered, content-based gesture disambiguation. We describe how an already existing framework, GestureAgents, implements this new approach, focusing on expanding the description of the relevant elements related to this problem, and conclude with some example applications and a discussion., The research leading to these results has received funding from the European Union Seventh Framework Programme FP7 / 2007–2013 through PHENICX project under grant agreement n° 601166.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

On the interrelation between listener characteristics and the perception of emotion in classical orchestra music

  • Schedl, Markus
  • Gómez Gutiérrez, Emilia, 1975-
  • Trent, Erika S.
  • Tkalčič, Marko
  • Eghbal-Zadeh, Hamid
  • Martorell Domínguez, Agustín
This study deals with the strong relationship between emotions and music, investigating three main research questions: (RQ1) Are there differences in human music perception (e.g., emotions, tempo, instrumentation, and complexity), according to musical education, experience, demographics, and personality traits?; (RQ2) Do certain perceived music characteristics correlate (e.g., tension and sadness), irrespective of a particular listener's background or personality?; (RQ3) Does human perception of music characteristics, such as emotions and tempo, correlate with descriptors extracted from music audio signals? To investigate our research questions, we conducted two user studies focusing on different groups of subjects. We used web-based surveys to collect information about demographics, listening habits, musical education, personality, and perceptual ratings with respect to perceived emotions, tempo, complexity, and instrumentation for 15 segments of Beethoven's 3rd symphony, “Eroica”. Our experiments showed that all three research questions can be affirmed, at least partly. We found strong support for RQ2 and RQ3, while RQ1 could be confirmed only for some perceptual aspects and user groups., This research is supported by the EU-FP7 project no. 601166 (“PHENICX”), the Austrian Science Fund (FWF): P25655 and the Spanish Government (CASAS project TIN2015-70816-R).
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Evaluation and combination of pitch estimation methods for melody extraction in symphonic classical music

  • Bosch, Juan J.
  • Marxer Piñón, Ricard
  • Gómez Gutiérrez, Emilia, 1975-
The extraction of pitch information is arguably one of the most important tasks in automatic music description systems. However, previous research and evaluation datasets dealing with pitch estimation focused on relatively limited kinds of musical data. This work aims to broaden this scope by addressing symphonic western classical music recordings, focusing on pitch estimation for melody extraction. This material is characterized by a high number of overlapping sources, and by the fact that the melody may be played by different instrumental sections, often alternating within an excerpt. We evaluate the performance of eleven state-of-the-art pitch salience functions, multipitch estimation and melody extraction algorithms when determining the sequence of pitches corresponding to the main melody in a varied set of pieces. An important contribution of the present study is the proposed evaluation framework, including the annotation methodology, generated dataset and evaluation metrics. The results show that the assumptions made by certain methods hold better than others when dealing with this type of music signal, leading to a better performance. Additionally, we propose a simple method for combining the output of several algorithms, with promising results., This work is supported by the European Union Seventh Framework Programme FP7 / 2007-2013 through the PHENICX project (grant agreement no. 601166). This work is also (partly) supported by the Spanish Ministry of Economy and Competitiveness/nunder the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

A personality-based adaptive system for visualizing classical music performances

  • Schedl, Markus
  • Melenhorst, Mark
  • Liem, Cynthia C. S.
  • Martorell Domínguez, Agustín
  • Mayor, Oscar
  • Tkalčič, Marko
To enhance the experience of listening to classical orchestra music, either in the concert hall or at home, we present a personalized system that integrates three visualization/interaction concepts: Score Follower (points to the current position in the score), Orchestra Layout (illustrates instruments that are currently playing and their dynamics), and Structure Visualization (visualizes structural elements such as themes or motifs). Motivated by previous literature that found evidence for connections between personality and music consumption and preference, we first assessed in a user study to which extent personality traits and music visualization preferences correlate. Measuring preference via pragmatic quality and personality traits according to the Big Five Inventory (BFI) questionnaire, we found substantial interconnections between them. These translate into rules relating certain personality traits (e.g., extraversion or agreeableness) to preference rankings of the visualizations. In the proposed personality-based system, users are grouped into four clusters according to their answers to the most significant personality questions determined in the study. The order of the visualizations for a given user is adapted with respect to the ranking preferred by other users in the same cluster. Evaluation of the system was carried out by a second user study that showed a significantly higher normalized discounted cumulative gain (NDCG) for the personalized system in comparison to a system with randomized order of the visualizations., This work is supported by the EU FP7-ICT-2011-9 project 601166 (“PHENICX”) and the Austrian Science Fund (FWF): P25655.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Music information retrieval: recent developments and applications

  • Schedl, Markus
  • Gómez Gutiérrez, Emilia, 1975-
  • Urbano, Julián
We provide a survey of the field of Music Information Retrieval (MIR), in particular paying attention to latest developments, such as semantic auto-tagging and user-centric retrieval and recommendation approaches. We first elaborate on well-established and proven methods for feature extraction and music indexing, from both the audio signal and contextual data sources about music items, such as web pages or collaborative tags. These in turn enable a wide variety of music retrieval tasks, such as semantic music search or music identification (“query by example”). Subsequently, we review current work on user analysis and modeling in the context of music recommendation and retrieval, addressing the recent trend towards user-centric and adaptive approaches and systems. A discussion follows about the important aspect of how various MIR approaches to different problems are evaluated and compared. Eventually, a discussion about the major open challenges concludes the survey., This work is supported by the Austrian Science Fund (FWF): P22856 and P25655, as well as by the European Union Seventh Framework Programme FP7 / 2007-2013 through the PHENICX project under grant agreement no. 601166, the A4U postdoctoral grants programme, and the Junta of Andalusia through the COFLA2 project (P12-TIC-1362). The authors would further like to thank Masataka Goto, Mohamed Sordo, and Edith Law for granting permission to include their visual material in this survey, and Juan J. Bosch for his comments on the manuscript. Furthermore, the authors would like to express their gratitude to the anonymous reviewers for their highly valuable suggestions for improving the manuscript.