Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 28
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Digital modeling of bridge driving-point admittance from measurements on violin-family instruments

  • Maestre Gómez, Esteban
  • Scavone, Gary P.
  • Smith, Julius O.
We present a methodology for digital modeling of D-dimensional driving-point bridge admittances from vibration measurements on instruments of the violin family. Our study, centered around the two-dimensional case for violin, viola, and cello, is based on using the modal framework to construct an admittance formulation providing physically meaningful and effective control over model parameters. In a first stage, mode frequencies and bandwidths are estimated in the frequency domain via solving a non-convex, constrained optimization problem. Then, mode amplitudes are estimated via semidefinite programming while enforcing passivity. We obtain accurate, low-order digital admittance models suited for real-time sound synthesis via physical models., This work was partially funded by the Catalan Gov. through a Beatriu de Pin´os Fellowship
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Measuring ensemble interdependence in a string quartet through analysis of multidimensional performance data

  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Maestre Gómez, Esteban
In a musical ensemble such as a string quartet, the musicians interact and influence each other's actions in several aspects of the performance simultaneously in order to achieve a common aesthetic goal. In this article, we present and evaluate a computational approach for measuring the degree to which these interactions exist in a given performance. We recorded a number of string quartet exercises under two experimental conditions (solo and ensemble), acquiring both audio and bowing motion data. Numerical features in the form of time series were extracted from the data as performance descriptors representative of four distinct dimensions of the performance: Intonation, Dynamics, Timbre, and Tempo. Four different interdependence estimation methods (two linear and two nonlinear) were applied to the extracted features in order to assess the overall level of interdependence between the four musicians. The obtained results suggest that it is possible to correctly discriminate between the two experimental conditions by quantifying interdependence between the musicians in each of the studied performance dimensions; the nonlinear methods appear to perform best for most of the numerical features tested. Moreover, by using the solo recordings as a reference to which the ensemble recordings are contrasted, it is feasible to compare the amount of interdependence that is established between the musicians in a given performance dimension across all exercises, and relate the results to the underlying goal of the exercise. We discuss our findings in the context of ensemble performance research, the current limitations of our approach, and the ways in which it can be expanded and consolidated., This work was partially supported by the EU FP7 FET-Open SIEMPRE Project no. FP7-ICT-2009-C-250026, the Catalan Research Funding Agency AGAUR, Beatriu de Pinos Grant 2010 BP-A 00209, and FP7 Marie Curie Grant no. PIOF-GA-2012-331270. We would like to thank Erika Donald and Marcelo Wanderley, as well as Carolina Brum Medeiros, Vincent Freour, and Marcello Giordano for the support in organizing the experiment, and the CIRMMT and BRAMS labs at Montreal, Quebec, Canada for hosting them. Finally, we would like to thank the reviewers for their thoughtful comments as well as for pointing out errors/omissions in the manuscript.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Aural-based detection and assessment of real versus artificially synchronized string quartet performance

  • Papiotis, Panagiotis, 1985-
  • Herrera Boyer, Perfecto, 1964-
  • Marchini, Marco, 1984-
  • Maestre Gómez, Esteban
Comunicació presentada a: the 3rd International Conference on Music & Emotion (ICME3), celebrada a Jyväskylä, Finlàndia, de l'11 al 15 de juny de 2013., In a musical ensemble musicians can influence each other’s performance in terms not only of timing but also in other aspects of the performance such as dynamics, intonation, and timbre. The goal of this work is to test whether this influence can be perceived by a listener from an audio recording solely. We utilize a set of string quartet recordings where every piece is recorded in two experimental conditions: the solo condition, where each musician performs alone; and the ensemble condition, where the musicians perform together after a brief rehearsal. Using state-of-the-art audio analysis/synthesis methods, we artificially synchronize the record-ings in the solo condition note-by-note, thus generating a set of pseudo-ensemble performances where there is no interaction between the musicians. We then carry out a series of listening tests: first, the subjects are tasked with comparing the quality of the performance and the degree of coordination for the two recordings, without knowing that one of them is artificially synchronized. Then, we reveal to the listeners that one of the two versions is artificially synchronized and ask them to point out which recording is which. The results sug-gest that listeners cannot easily discriminate between the real and artificially synchronized recordings; fur-thermore, the accuracy of their judgements appears to be affected by the listeners' level of musical training as well as the piece that is performed., The work presented on this document has been partially supported by the EU-FP7 FET SIEMPRE project and an AGAUR research grant from Generalitat de Catalunya.
Repositorio Digital de la UPF
Publication Capítulo o parte de un libro (BookPart)

Measuring ensemble synchrony through violin performance parameters: a preliminary progress report

  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Maestre Gómez, Esteban
  • Pérez Carrillo, Alfonso Antonio, 1977-
In this article we present our ongoing work on expressive /nperformance analysis for violin and string ensembles, in terms of synchronization in intonation, timing, dynamics and articulation. Our current /nresearch objectives are outlined, along with an overview for the methods used /nto achieve them; finally, focusing on the case of intonation synchronization in /nviolin duets, some preliminary results and conclusions based on experimental /nrecordings are discussed., The work presented on this document has been partially supported by the EU-FP7 /nICT SIEMPRE project.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Investigating the relationship between expressivity and synchronization in ensemble performance: an exploratory study

  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Maestre Gómez, Esteban
Comunicació presentada al International Symposium on Performance Science (ISPS), celebrat a Viena (Àustria) els dies 28 a 31 d'agost de 2013., We present an exploratory study on ensemble expressive performance based on the analysis of string quartet recordings. We recorded a piece with three expressive intentions: mechanical, normal, and exaggerated. We made use of bowing gesture data (bow velocity and force) acquired through a motion tracking system to devise a precise score performance alignment. Individual contact microphone audio signals allowed extraction of a set of audio descriptors for each musician and each note. We show how tempo and loudness on a macro-scale changed across expressive intentions and score sections. The score is also taken into account in the analysis by extracting contextual attributes for each note. We show that micro-deviations were affected by note contextual attributes, whereas the effect of expressive intention varied across sections. We find sections that exhibited a lower entrainment, where individual parts tended to be freer and presented more asynchronies., This work was supported by EU FET-Open SIEMPRE and by SIEMPRE-MAS4.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

repoVizz: a framework for remote storage, browsing, annotation, and exchange of multi-modal data

  • Mayor, Oscar
  • Llimona Torras, Quim
  • Marchini, Marco, 1984-
  • Papiotis, Panagiotis, 1985-
  • Maestre Gómez, Esteban
In this technical demo we present repoVizz (http://repovizz.upf.edu), an integrated online system capable of structural formatting and remote storage, browsing, exchange, annotation, and visualization of synchronous multi-modal, time-aligned data. Motivated by a growing need for data-driven collaborative research, repoVizz aims to resolve commonly encountered difficulties in sharing or browsing large collections of multi-modal data. At its current state, repoVizz is designed to hold time-aligned streams of heterogeneous data: audio, video, motion capture, physiological signals, extracted descriptors, annotations, et cetera. Most popular formats for audio and video are supported, while Broadcast WAVE or CSV formats are adopted for streams other than audio or video (e.g., motion capture or physiological signals). The data itself is structured via customized XML files, allowing the user to (re-) organize multi-modal data in any hierarchical manner, as the XML structure only holds metadata and pointers to data files. Datasets are stored in an online database, allowing the user to interact with the data remotely through a powerful HTML5 visual interface accessible from any standard web browser; this feature can be considered a key aspect of repoVizz since data can be explored, annotated, or visualized from any location or device. Data exchange and upload/download is made easy and secure via a number of data conversion tools and a user/permission management system., This work was partially supported by the EU FET-Open/nProject SIEMPRE (FP7-ICT-2009-C-250026) and the Catalan Government through an AGAUR Fellowship and TECNIO/ngrant (TECCIT12-1-0003) from ACC1Ó.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Synchronization of intonation adjustments in violin duets: towards an objective evaluation of musical interaction

  • Papiotis, Panagiotis, 1985-
  • Maestre Gómez, Esteban
  • Marchini, Marco, 1984-
  • Pérez Carrillo, Alfonso Antonio, 1977-
In ensemble music performance, such as a string quartet or du et, the musicians interact and influence each other’s performance via a multitude of parameters – including tempo, dynamics, articulation of musical phrases and, depending on the type of instrument, intonation. This paper presents our ongoing research on the effect of interaction between violinists, in terms of intonation. We base our analysis on a series of experiments with professional as well as amateur musicians playing in duet and solo experimental set-ups, and then apply a series of interdependence measures on each violinist’s pitch deviations from the score. Our results show that while it is possible to, solely based on intonation, distinguish between solo and duet performances for simple cases, there is a multitude of underlying factors that need to be analyzed before these techniques can be applied to more complex pieces and/or non-experimental situations., The work presented on this document has been partially supported by the EU-FP7 ICT SIEMPRE project.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Enriched multimodal representations of music performances: online access and visualization

  • Maestre Gómez, Esteban
  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Llimona Torras, Quim
  • Mayor, Oscar
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Wanderley, Marcelo
The authors provide a first-person outlook on the technical challenges involved in the recording, analysis, archiving, and cloud-based interchange of multimodal string quartet performance data as part of a collaborative research project on ensemble music making. To facilitate the sharing of their own collection of multimodal recordings and extracted descriptors and annotations, they developed a hosting platform through which multimodal data (audio, video, motion capture, and derived signals) can be stored, visualized, annotated, and selectively retrieved via a web interface and a dedicated API. This article offers a twofold contribution: the authors open their collection of enriched multimodal recordings, the Quartet dataset, to the community, and they introduce and enable access to their multimodal data exchange platform and web application, the Repovizz system. This article is part of a special issue on multimedia technologies for enriched music., This work was partially funded by the EU FP7 project numbers 250026 (SIEMPRE) and 601166 (PHENICX), the EU Marie Curie Integration Grant PIOF-GA-2012-331270, and the AGAUR Catalan Public Funding Agency. Moreover, the authors thank the BRAMS Laboratory and the contributions by Erika Donald, Marcello Giordano, Carolina B. Medeiros, Marina Azón, Guillermo Navarro, and Jordi Llop.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Measuring ensemble interdependence in a string quartet through analysis of multidimensional performance data

  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Maestre Gómez, Esteban
In a musical ensemble such as a string quartet, the musicians interact and influence each other's actions in several aspects of the performance simultaneously in order to achieve a common aesthetic goal. In this article, we present and evaluate a computational approach for measuring the degree to which these interactions exist in a given performance. We recorded a number of string quartet exercises under two experimental conditions (solo and ensemble), acquiring both audio and bowing motion data. Numerical features in the form of time series were extracted from the data as performance descriptors representative of four distinct dimensions of the performance: Intonation, Dynamics, Timbre, and Tempo. Four different interdependence estimation methods (two linear and two nonlinear) were applied to the extracted features in order to assess the overall level of interdependence between the four musicians. The obtained results suggest that it is possible to correctly discriminate between the two experimental conditions by quantifying interdependence between the musicians in each of the studied performance dimensions; the nonlinear methods appear to perform best for most of the numerical features tested. Moreover, by using the solo recordings as a reference to which the ensemble recordings are contrasted, it is feasible to compare the amount of interdependence that is established between the musicians in a given performance dimension across all exercises, and relate the results to the underlying goal of the exercise. We discuss our findings in the context of ensemble performance research, the current limitations of our approach, and the ways in which it can be expanded and consolidated., This work was partially supported by the EU FP7 FET-Open SIEMPRE Project no. FP7-ICT-2009-C-250026, the Catalan Research Funding Agency AGAUR, Beatriu de Pinos Grant 2010 BP-A 00209, and FP7 Marie Curie Grant no. PIOF-GA-2012-331270. We would like to thank Erika Donald and Marcelo Wanderley, as well as Carolina Brum Medeiros, Vincent Freour, and Marcello Giordano for the support in organizing the experiment, and the CIRMMT and BRAMS labs at Montreal, Quebec, Canada for hosting them. Finally, we would like to thank the reviewers for their thoughtful comments as well as for pointing out errors/omissions in the manuscript.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Multidimensional analysis of interdependence in a string quartet

  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Maestre Gómez, Esteban
Comunicació presentada al International Symposium on Performance Science (ISPS), celebrat a Viena (Àustria) els dies 28 a 31 d'agost de 2013., In a musical ensemble such as a string quartet, the performers can influence each other’s actions in several aspects of the performance simultaneously. Based on a set of recorded string quartet exercises, we carried out a quantitative analysis of ensemble interdependence in four distinct dimensions of the performance: dynamics, intonation, tempo, and timbre. We investigated the fluctuations of interdependence across these four dimensions, and in relation to the exercise being performed. Our findings suggest that, although certain differences can be observed between the four dimensions, the most influential factor on ensemble interdependence is the musical task, shaped by the underlying score., The work presented on this document has been partially supported by the EU-FP7 FET SIEMPRE project and an AGAUR research grant from Generalitat de Catalunya.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

RepoVizz: a multi-modal on-line database and browsing tool for music performance research

  • Mayor, Oscar
  • Llop, Jordi
  • Maestre Gómez, Esteban
RepoVizz is a data repository and visualization tool for/nstructured storage and user-friendly browsing of music performance/nmulti-modal recordings. The primary purpose of/nRepoVizz is to offer means for the scientific community to/ngain on-line access to a music performance multi-modal database/nshared among researchers. RepoVizz is designed to/nhold synchronized streams of heterogeneous data (audio,/nvideo, motion capture, physiological measures, extracted/ndescriptors, etc.), annotations, and musical scores. Data/nstreams are stored as single-channel PCM files. The motion-/nsequence format MJ2 is chosen for video. Annotations/nare stored as text files. MusicXML is used for musical/nscores. Data is structured by customizable XML skeleton/nfiles enabling meaningful retrieval. Skeleton files are first/ncreated during data gathering, and provide means to (re-/n)organize acquired data in any desired hierarchical structure,/nas they only hold pointers to stored data files. Once a/ndataset is created and uploaded to the server, each skeleton/nfile defines a view. Multi-track data visualization is done/nremotely via a powerful HTML5-based environment that/nenables web-driven editing (add annotations, extract descriptors)/nand downloading of datasets. A first instance of/nRepoVizz has been created within the EU FET-Open/nProject SIEMPRE, devoted to studying social interaction in/nensemble performance.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Computational analysis of solo versus ensemble performance in string quartets: intonation and dynamics

  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Maestre Gómez, Esteban
Comunicació presentada a la conferència conjunta que inclou la 12th International Conference on Music Perception and Cognition (ICMPC) i la 8th Triennial Conference of the European Society, celebrada a Tessalònica (Grècia) els dies 23 a 28 de juliol de 2012., Musical ensembles, such as a string quartet, are a clear case of music performance where a joint interpretation of the score as well as joint action during the performance is required by the musicians. Of the several explicit and implicit ways through which the musicians cooperate, we focus on the acoustic result of the performance – in this case in terms of dynamics and intonation - and attempt to detect evidence of interdependence among the musicians by performing a computational analysis. We have recorded a set of string quartet exercises whose challenge lies in achieving ensemble cohesion rather than correctly performing one’s individual task successfully, which serve as a ‘ground truth’ dataset; these exercises were recorded by a professional string quartet in two experimental conditions: solo, where each musician performs their part alone without having access to the full quartet score, and ensemble, where the musicians perform the exercise together following a short rehearsal period. Through an automatic analysis and post-processing of audio and motion capture data, we extract a set of low-level features, on which we apply several numerical methods of interdependence (such as Pearson correlation, Mutual Information, Granger causality, and Nonlinear coupling) in order to measure the interdependence -or lack thereofamong the musicians during the performance. Results show that, although dependent on the underlying musical score, this methodology can be used in order to automatically analyze the performance of a musical ensemble., The work presented on this document has been partially supported by the EU-FP7 FET SIEMPRE project and an AGAUR research grant from Generalitat de Catalunya.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

The Sense of ensemble: a machine learning approach to expressive performance modelling in string quartets

  • Marchini, Marco, 1984-
  • Ramírez, Rafael, 1966-
  • Maestre Gómez, Esteban
Computational approaches for modelling expressive music performance have produced systems that emulate music expression, but few steps have been taken in the domain of ensemble performance. In this paper, we propose a novel method for building computational models of ensemble expressive performance and show how this method can be applied for deriving new insights about collaboration among musicians. In order to address the problem of inter-dependence among musicians we propose the introduction of inter-voice contextual attributes. We evaluate the method on data extracted from multi-modal recordings of string quartet performances in two different conditions: solo and ensemble. We used machine-learning algorithms to produce computational models for predicting intensity, timing deviations, vibrato extent, and bowing speed of each note. As a result, the introduced inter-voice contextual attributes generally improved the prediction of the expressive parameters. Furthermore, results on attribute selection show that the models trained on ensemble recordings took more advantage of inter-voice contextual attributes than those trained on solo recordings., This work was partially supported by the EU FP7 FET-Open SIEMPRE Project (FP7-ICT- 2009-C-250026), by the Spanish TIN project DRIMS (TIN2009-14274-C02-01), and by the Catalan Research Funding Agency AGAUR.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Timing synchronization in string quartet performance: a preliminary study

  • Marchini, Marco, 1984-
  • Papiotis, Panagiotis, 1985-
  • Maestre Gómez, Esteban
This work presents a preliminary study of timing synchronization/nphenomena in string quartet performance. Accurate timing information/nextracted from real recordings is used to compare timing deviations/nin solo and ensemble performance when executing a simple musical/npassage. Multi-modal data is acquired from real performance and/nprocessed towards obtaining note-level segmentation of recorded performances./nFrom such segmentation, a series of timing deviation analyses/nare carried out at two different temporal levels, focusing on the exploration/nof significant differences between solo and ensemble performances./nThis paper briefly introduces, via an initial exploratory study, the experimental/nframework on which further, more complete analyses are to be/ncarried out with the aim of observing and describing certain synchronization/nphenomena taking place in ensemble music making., Papiotis, Panagiotis
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Digital modeling of bridge driving-point admittance from measurements on violin-family instruments

  • Maestre Gómez, Esteban
  • Scavone, Gary P.
  • Smith, Julius O.
We present a methodology for digital modeling of D-dimensional driving-point bridge admittances from vibration measurements on instruments of the violin family. Our study, centered around the two-dimensional case for violin, viola, and cello, is based on using the modal framework to construct an admittance formulation providing physically meaningful and effective control over model parameters. In a first stage, mode frequencies and bandwidths are estimated in the frequency domain via solving a non-convex, constrained optimization problem. Then, mode amplitudes are estimated via semidefinite programming while enforcing passivity. We obtain accurate, low-order digital admittance models suited for real-time sound synthesis via physical models., This work was partially funded by the Catalan Gov. through a Beatriu de Pin´os Fellowship
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Enriched multimodal representations of music performances: online access and visualization

  • Maestre Gómez, Esteban
  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Llimona Torras, Quim
  • Mayor, Oscar
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Wanderley, Marcelo
The authors provide a first-person outlook on the technical challenges involved in the recording, analysis, archiving, and cloud-based interchange of multimodal string quartet performance data as part of a collaborative research project on ensemble music making. To facilitate the sharing of their own collection of multimodal recordings and extracted descriptors and annotations, they developed a hosting platform through which multimodal data (audio, video, motion capture, and derived signals) can be stored, visualized, annotated, and selectively retrieved via a web interface and a dedicated API. This article offers a twofold contribution: the authors open their collection of enriched multimodal recordings, the Quartet dataset, to the community, and they introduce and enable access to their multimodal data exchange platform and web application, the Repovizz system. This article is part of a special issue on multimedia technologies for enriched music., This work was partially funded by the EU FP7 project numbers 250026 (SIEMPRE) and 601166 (PHENICX), the EU Marie Curie Integration Grant PIOF-GA-2012-331270, and the AGAUR Catalan Public Funding Agency. Moreover, the authors thank the BRAMS Laboratory and the contributions by Erika Donald, Marcello Giordano, Carolina B. Medeiros, Marina Azón, Guillermo Navarro, and Jordi Llop.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Timing synchronization in string quartet performance: a preliminary study

  • Marchini, Marco, 1984-
  • Papiotis, Panagiotis, 1985-
  • Maestre Gómez, Esteban
This work presents a preliminary study of timing synchronization/nphenomena in string quartet performance. Accurate timing information/nextracted from real recordings is used to compare timing deviations/nin solo and ensemble performance when executing a simple musical/npassage. Multi-modal data is acquired from real performance and/nprocessed towards obtaining note-level segmentation of recorded performances./nFrom such segmentation, a series of timing deviation analyses/nare carried out at two different temporal levels, focusing on the exploration/nof significant differences between solo and ensemble performances./nThis paper briefly introduces, via an initial exploratory study, the experimental/nframework on which further, more complete analyses are to be/ncarried out with the aim of observing and describing certain synchronization/nphenomena taking place in ensemble music making., Papiotis, Panagiotis
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

RepoVizz: a multi-modal on-line database and browsing tool for music performance research

  • Mayor, Oscar
  • Llop, Jordi
  • Maestre Gómez, Esteban
RepoVizz is a data repository and visualization tool for/nstructured storage and user-friendly browsing of music performance/nmulti-modal recordings. The primary purpose of/nRepoVizz is to offer means for the scientific community to/ngain on-line access to a music performance multi-modal database/nshared among researchers. RepoVizz is designed to/nhold synchronized streams of heterogeneous data (audio,/nvideo, motion capture, physiological measures, extracted/ndescriptors, etc.), annotations, and musical scores. Data/nstreams are stored as single-channel PCM files. The motion-/nsequence format MJ2 is chosen for video. Annotations/nare stored as text files. MusicXML is used for musical/nscores. Data is structured by customizable XML skeleton/nfiles enabling meaningful retrieval. Skeleton files are first/ncreated during data gathering, and provide means to (re-/n)organize acquired data in any desired hierarchical structure,/nas they only hold pointers to stored data files. Once a/ndataset is created and uploaded to the server, each skeleton/nfile defines a view. Multi-track data visualization is done/nremotely via a powerful HTML5-based environment that/nenables web-driven editing (add annotations, extract descriptors)/nand downloading of datasets. A first instance of/nRepoVizz has been created within the EU FET-Open/nProject SIEMPRE, devoted to studying social interaction in/nensemble performance.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Measuring ensemble interdependence in a string quartet through analysis of multidimensional performance data

  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Maestre Gómez, Esteban
In a musical ensemble such as a string quartet, the musicians interact and influence each other's actions in several aspects of the performance simultaneously in order to achieve a common aesthetic goal. In this article, we present and evaluate a computational approach for measuring the degree to which these interactions exist in a given performance. We recorded a number of string quartet exercises under two experimental conditions (solo and ensemble), acquiring both audio and bowing motion data. Numerical features in the form of time series were extracted from the data as performance descriptors representative of four distinct dimensions of the performance: Intonation, Dynamics, Timbre, and Tempo. Four different interdependence estimation methods (two linear and two nonlinear) were applied to the extracted features in order to assess the overall level of interdependence between the four musicians. The obtained results suggest that it is possible to correctly discriminate between the two experimental conditions by quantifying interdependence between the musicians in each of the studied performance dimensions; the nonlinear methods appear to perform best for most of the numerical features tested. Moreover, by using the solo recordings as a reference to which the ensemble recordings are contrasted, it is feasible to compare the amount of interdependence that is established between the musicians in a given performance dimension across all exercises, and relate the results to the underlying goal of the exercise. We discuss our findings in the context of ensemble performance research, the current limitations of our approach, and the ways in which it can be expanded and consolidated., This work was partially supported by the EU FP7 FET-Open SIEMPRE Project no. FP7-ICT-2009-C-250026, the Catalan Research Funding Agency AGAUR, Beatriu de Pinos Grant 2010 BP-A 00209, and FP7 Marie Curie Grant no. PIOF-GA-2012-331270. We would like to thank Erika Donald and Marcelo Wanderley, as well as Carolina Brum Medeiros, Vincent Freour, and Marcello Giordano for the support in organizing the experiment, and the CIRMMT and BRAMS labs at Montreal, Quebec, Canada for hosting them. Finally, we would like to thank the reviewers for their thoughtful comments as well as for pointing out errors/omissions in the manuscript.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Investigating the relationship between expressivity and synchronization in ensemble performance: an exploratory study

  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Maestre Gómez, Esteban
Comunicació presentada al International Symposium on Performance Science (ISPS), celebrat a Viena (Àustria) els dies 28 a 31 d'agost de 2013., We present an exploratory study on ensemble expressive performance based on the analysis of string quartet recordings. We recorded a piece with three expressive intentions: mechanical, normal, and exaggerated. We made use of bowing gesture data (bow velocity and force) acquired through a motion tracking system to devise a precise score performance alignment. Individual contact microphone audio signals allowed extraction of a set of audio descriptors for each musician and each note. We show how tempo and loudness on a macro-scale changed across expressive intentions and score sections. The score is also taken into account in the analysis by extracting contextual attributes for each note. We show that micro-deviations were affected by note contextual attributes, whereas the effect of expressive intention varied across sections. We find sections that exhibited a lower entrainment, where individual parts tended to be freer and presented more asynchronies., This work was supported by EU FET-Open SIEMPRE and by SIEMPRE-MAS4.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Measuring ensemble interdependence in a string quartet through analysis of multidimensional performance data

  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Maestre Gómez, Esteban
In a musical ensemble such as a string quartet, the musicians interact and influence each other's actions in several aspects of the performance simultaneously in order to achieve a common aesthetic goal. In this article, we present and evaluate a computational approach for measuring the degree to which these interactions exist in a given performance. We recorded a number of string quartet exercises under two experimental conditions (solo and ensemble), acquiring both audio and bowing motion data. Numerical features in the form of time series were extracted from the data as performance descriptors representative of four distinct dimensions of the performance: Intonation, Dynamics, Timbre, and Tempo. Four different interdependence estimation methods (two linear and two nonlinear) were applied to the extracted features in order to assess the overall level of interdependence between the four musicians. The obtained results suggest that it is possible to correctly discriminate between the two experimental conditions by quantifying interdependence between the musicians in each of the studied performance dimensions; the nonlinear methods appear to perform best for most of the numerical features tested. Moreover, by using the solo recordings as a reference to which the ensemble recordings are contrasted, it is feasible to compare the amount of interdependence that is established between the musicians in a given performance dimension across all exercises, and relate the results to the underlying goal of the exercise. We discuss our findings in the context of ensemble performance research, the current limitations of our approach, and the ways in which it can be expanded and consolidated., This work was partially supported by the EU FP7 FET-Open SIEMPRE Project no. FP7-ICT-2009-C-250026, the Catalan Research Funding Agency AGAUR, Beatriu de Pinos Grant 2010 BP-A 00209, and FP7 Marie Curie Grant no. PIOF-GA-2012-331270. We would like to thank Erika Donald and Marcelo Wanderley, as well as Carolina Brum Medeiros, Vincent Freour, and Marcello Giordano for the support in organizing the experiment, and the CIRMMT and BRAMS labs at Montreal, Quebec, Canada for hosting them. Finally, we would like to thank the reviewers for their thoughtful comments as well as for pointing out errors/omissions in the manuscript.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Aural-based detection and assessment of real versus artificially synchronized string quartet performance

  • Papiotis, Panagiotis, 1985-
  • Herrera Boyer, Perfecto, 1964-
  • Marchini, Marco, 1984-
  • Maestre Gómez, Esteban
Comunicació presentada a: the 3rd International Conference on Music & Emotion (ICME3), celebrada a Jyväskylä, Finlàndia, de l'11 al 15 de juny de 2013., In a musical ensemble musicians can influence each other’s performance in terms not only of timing but also in other aspects of the performance such as dynamics, intonation, and timbre. The goal of this work is to test whether this influence can be perceived by a listener from an audio recording solely. We utilize a set of string quartet recordings where every piece is recorded in two experimental conditions: the solo condition, where each musician performs alone; and the ensemble condition, where the musicians perform together after a brief rehearsal. Using state-of-the-art audio analysis/synthesis methods, we artificially synchronize the record-ings in the solo condition note-by-note, thus generating a set of pseudo-ensemble performances where there is no interaction between the musicians. We then carry out a series of listening tests: first, the subjects are tasked with comparing the quality of the performance and the degree of coordination for the two recordings, without knowing that one of them is artificially synchronized. Then, we reveal to the listeners that one of the two versions is artificially synchronized and ask them to point out which recording is which. The results sug-gest that listeners cannot easily discriminate between the real and artificially synchronized recordings; fur-thermore, the accuracy of their judgements appears to be affected by the listeners' level of musical training as well as the piece that is performed., The work presented on this document has been partially supported by the EU-FP7 FET SIEMPRE project and an AGAUR research grant from Generalitat de Catalunya.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Synchronization of intonation adjustments in violin duets: towards an objective evaluation of musical interaction

  • Papiotis, Panagiotis, 1985-
  • Maestre Gómez, Esteban
  • Marchini, Marco, 1984-
  • Pérez Carrillo, Alfonso Antonio, 1977-
In ensemble music performance, such as a string quartet or du et, the musicians interact and influence each other’s performance via a multitude of parameters – including tempo, dynamics, articulation of musical phrases and, depending on the type of instrument, intonation. This paper presents our ongoing research on the effect of interaction between violinists, in terms of intonation. We base our analysis on a series of experiments with professional as well as amateur musicians playing in duet and solo experimental set-ups, and then apply a series of interdependence measures on each violinist’s pitch deviations from the score. Our results show that while it is possible to, solely based on intonation, distinguish between solo and duet performances for simple cases, there is a multitude of underlying factors that need to be analyzed before these techniques can be applied to more complex pieces and/or non-experimental situations., The work presented on this document has been partially supported by the EU-FP7 ICT SIEMPRE project.
Recercat. Dipósit de la Recerca de Catalunya
Publication Capítulo o parte de un libro (BookPart)

Measuring ensemble synchrony through violin performance parameters: a preliminary progress report

  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Maestre Gómez, Esteban
  • Pérez Carrillo, Alfonso Antonio, 1977-
In this article we present our ongoing work on expressive /nperformance analysis for violin and string ensembles, in terms of synchronization in intonation, timing, dynamics and articulation. Our current /nresearch objectives are outlined, along with an overview for the methods used /nto achieve them; finally, focusing on the case of intonation synchronization in /nviolin duets, some preliminary results and conclusions based on experimental /nrecordings are discussed., The work presented on this document has been partially supported by the EU-FP7 /nICT SIEMPRE project.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

The Sense of ensemble: a machine learning approach to expressive performance modelling in string quartets

  • Marchini, Marco, 1984-
  • Ramírez, Rafael, 1966-
  • Maestre Gómez, Esteban
Computational approaches for modelling expressive music performance have produced systems that emulate music expression, but few steps have been taken in the domain of ensemble performance. In this paper, we propose a novel method for building computational models of ensemble expressive performance and show how this method can be applied for deriving new insights about collaboration among musicians. In order to address the problem of inter-dependence among musicians we propose the introduction of inter-voice contextual attributes. We evaluate the method on data extracted from multi-modal recordings of string quartet performances in two different conditions: solo and ensemble. We used machine-learning algorithms to produce computational models for predicting intensity, timing deviations, vibrato extent, and bowing speed of each note. As a result, the introduced inter-voice contextual attributes generally improved the prediction of the expressive parameters. Furthermore, results on attribute selection show that the models trained on ensemble recordings took more advantage of inter-voice contextual attributes than those trained on solo recordings., This work was partially supported by the EU FP7 FET-Open SIEMPRE Project (FP7-ICT- 2009-C-250026), by the Spanish TIN project DRIMS (TIN2009-14274-C02-01), and by the Catalan Research Funding Agency AGAUR.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

repoVizz: a framework for remote storage, browsing, annotation, and exchange of multi-modal data

  • Mayor, Oscar
  • Llimona Torras, Quim
  • Marchini, Marco, 1984-
  • Papiotis, Panagiotis, 1985-
  • Maestre Gómez, Esteban
In this technical demo we present repoVizz (http://repovizz.upf.edu), an integrated online system capable of structural formatting and remote storage, browsing, exchange, annotation, and visualization of synchronous multi-modal, time-aligned data. Motivated by a growing need for data-driven collaborative research, repoVizz aims to resolve commonly encountered difficulties in sharing or browsing large collections of multi-modal data. At its current state, repoVizz is designed to hold time-aligned streams of heterogeneous data: audio, video, motion capture, physiological signals, extracted descriptors, annotations, et cetera. Most popular formats for audio and video are supported, while Broadcast WAVE or CSV formats are adopted for streams other than audio or video (e.g., motion capture or physiological signals). The data itself is structured via customized XML files, allowing the user to (re-) organize multi-modal data in any hierarchical manner, as the XML structure only holds metadata and pointers to data files. Datasets are stored in an online database, allowing the user to interact with the data remotely through a powerful HTML5 visual interface accessible from any standard web browser; this feature can be considered a key aspect of repoVizz since data can be explored, annotated, or visualized from any location or device. Data exchange and upload/download is made easy and secure via a number of data conversion tools and a user/permission management system., This work was partially supported by the EU FET-Open/nProject SIEMPRE (FP7-ICT-2009-C-250026) and the Catalan Government through an AGAUR Fellowship and TECNIO/ngrant (TECCIT12-1-0003) from ACC1Ó.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Multidimensional analysis of interdependence in a string quartet

  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Maestre Gómez, Esteban
Comunicació presentada al International Symposium on Performance Science (ISPS), celebrat a Viena (Àustria) els dies 28 a 31 d'agost de 2013., In a musical ensemble such as a string quartet, the performers can influence each other’s actions in several aspects of the performance simultaneously. Based on a set of recorded string quartet exercises, we carried out a quantitative analysis of ensemble interdependence in four distinct dimensions of the performance: dynamics, intonation, tempo, and timbre. We investigated the fluctuations of interdependence across these four dimensions, and in relation to the exercise being performed. Our findings suggest that, although certain differences can be observed between the four dimensions, the most influential factor on ensemble interdependence is the musical task, shaped by the underlying score., The work presented on this document has been partially supported by the EU-FP7 FET SIEMPRE project and an AGAUR research grant from Generalitat de Catalunya.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Computational analysis of solo versus ensemble performance in string quartets: intonation and dynamics

  • Papiotis, Panagiotis, 1985-
  • Marchini, Marco, 1984-
  • Maestre Gómez, Esteban
Comunicació presentada a la conferència conjunta que inclou la 12th International Conference on Music Perception and Cognition (ICMPC) i la 8th Triennial Conference of the European Society, celebrada a Tessalònica (Grècia) els dies 23 a 28 de juliol de 2012., Musical ensembles, such as a string quartet, are a clear case of music performance where a joint interpretation of the score as well as joint action during the performance is required by the musicians. Of the several explicit and implicit ways through which the musicians cooperate, we focus on the acoustic result of the performance – in this case in terms of dynamics and intonation - and attempt to detect evidence of interdependence among the musicians by performing a computational analysis. We have recorded a set of string quartet exercises whose challenge lies in achieving ensemble cohesion rather than correctly performing one’s individual task successfully, which serve as a ‘ground truth’ dataset; these exercises were recorded by a professional string quartet in two experimental conditions: solo, where each musician performs their part alone without having access to the full quartet score, and ensemble, where the musicians perform the exercise together following a short rehearsal period. Through an automatic analysis and post-processing of audio and motion capture data, we extract a set of low-level features, on which we apply several numerical methods of interdependence (such as Pearson correlation, Mutual Information, Granger causality, and Nonlinear coupling) in order to measure the interdependence -or lack thereofamong the musicians during the performance. Results show that, although dependent on the underlying musical score, this methodology can be used in order to automatically analyze the performance of a musical ensemble., The work presented on this document has been partially supported by the EU-FP7 FET SIEMPRE project and an AGAUR research grant from Generalitat de Catalunya.