Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 54
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Phrase-Level modeling of expression in violin performances

  • Ortega, Fabio J. M.
  • Giraldo, Sergio
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Ramírez, Rafael, 1966-
Background: Expression is a key skill in music performance, and one that is difficult to address in music lessons. Computational models that learn from expert performances can help providing suggestions and feedback to students. Aim: We propose and analyze an approach to modeling variations in dynamics and note onset timing for solo violin pieces with the purpose of facilitating expressive performance learning in new pieces, for which no reference performance is available. Method: The method generates phrase–level predictions based on musical score information on the assumption that expressiveness is idiomatic, and thus influenced by similar–sounding melodies. Predictions were evaluated numerically using three different datasets and against note–level machine–learning models, and also perceptually by listeners, who were presented to synthesized versions of musical excerpts, and asked to choose the most human–sounding one. Some of the presented excerpts were synthesized to reflect the variations in dynamics and timing predicted by the model, whereas others were shaped to reflect the dynamics and timing of an actual expert performance, and a third group was presented with no expressive variations. Results: surprisingly, none of the three synthesized versions was consistently selected as human–like nor preferred with statistical significance by listeners. Possible interpretations of these results include the fact that the melodies might have been impossible to interpret outside their musical context, or that expressive features that were left out of the modeling such as note articulation and vibrato are, in fact, essential to the perception of expression in violin performance. Positive feedback by some listeners toward the modeled melodies in a blind setting indicate that the modeling approach was capable of generating appropriate renditions at least for a subset of the data. Numerically, performance in phrase–level suffers a small degradation if compared to note–level, but produces predictions easier to interpret visually, thus more useful in a pedagogical setting., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

A Real-time feedback learning tool to visualize sound quality in violin performances

  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
  • Waddell, George
  • Williamon, Aaron
Comunicació presentada a: 10th International Workshop on Machine Learning and Music (MML), celebrat a Barcelona (Espanya), el 6 d'octubre de 2017., The assessment of the sound properties of a performed mu- sical note has been widely studied in the past. Although a consensus exist on what is a good or a bad musical performance, there is not a formal de nition of performance tone quality due to its subjectivity. In this study we present a computational approach for the automatic assess- ment of violin sound production. We investigate the correlations among extracted features from audio performances and the perceptual quality of violin sounds rated by listeners using machine learning techniques. The obtained models are used for implementing a real-time feedback learning system., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), the European Union Horizon 2020 research and inno- vation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Musical neurofeedback for treating depression in elderly people

  • Ramírez, Rafael, 1966-
  • Palencia-Lefler Ors, Manuel
  • Giraldo, Sergio
  • Vamvakousis, Zacharias
We introduce a new neurofeedback approach, which allows users to manipulate expressive parameters in music performances using their emotional state, and we present the results of a pilot clinical experiment applying the approach to alleviate depression in elderly people. Ten adults (9 female and 1 male, mean = 84, SD = 5.8) with normal hearing participated in the neurofeedback study consisting of 10 sessions (2 sessions per week) of 15 min each. EEG data was acquired using the Emotiv EPOC EEG device. In all sessions, subjects were asked to sit in a comfortable chair facing two loudspeakers, to close their eyes, and to avoid moving during the experiment. Participants listened to music pieces preselected according to their music preferences, and were encouraged to increase the loudness and tempo of the pieces, based on their arousal and valence levels. The neurofeedback system was tuned so that increased arousal, computed as beta to alpha activity ratio in the frontal cortex corresponded to increased loudness, and increased valence, computed as relative frontal alpha activity in the right lobe compared to the left lobe, corresponded to increased tempo. Pre and post evaluation of six participants was performed using the BDI depression test, showing an average improvement of 17.2% (1.3) in their BDI scores at the end of the study. In addition, an analysis of the collected EEG data of the participants showed a significant decrease of relative alpha activity in their left frontal lobe (p = 0.00008), which may be interpreted as an improvement of their depression condition., This work has been partly sponsored by the Ministerio de Economia y Competitividad under Grant TIN2013-48152-C2-2-R (TIMuL Project).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Acquisition of violin instrumental gestures using an infrared depth camera

  • Vamvakousis, Zacharias
  • Perez, Alfonso
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 15th Sound and Music Computing Conference (SMC2018), celebrat del 4 al 7 de juliol a Limassol, Xipre., We present a low-cost six-degrees of freedom violin and bow pose-tracking system. The infrared and depth streams of an infrared-based Microsoft Kinect Mbox One depth camera are utilized to track the 3-dimensional position of 6 infrared markers attached on the violin and the bow. After computing the bow pose on the violin coordinate system, a number of bowing features are extracted. In order to evaluate the system’s performance, we recorded 4 bowing exercises using simultaneously our system along with a commercial two-sensor 3D tracking system based on electromagnetic field (EMF) sensing. The mean pearson coefficient values were 0.996 for the bow position, 0.889 for the bow velocity, 0.966 for the bow tilt, 0.692 for the bowbridge distance and 0.998 for the bow inclination. Compared to existing bowing-tracking solutions, the proposed solution might be appealing because of its low cost, easy setup and high performance., This work was partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), and the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

A Machine learning approach to ornamentation modeling and synthesis in jazz guitar

  • Ramírez, Rafael, 1966-
  • Giraldo, Sergio
We present a machine learning approach to automatically generate expressive (ornamented) jazz performances from un-expressive music scores. Features extracted from the scores and the corresponding audio recordings performed by a professional guitarist were used to train computational models for predicting melody ornamentation. As a first step, several machine learning techniques were explored to induce regression models for timing, onset, and dynamics (i.e. note duration and energy) transformations, and an ornamentation model for classifying notes as ornamented or non-ornamented. In a second step, the most suitable ornament for predicted ornamented notes was selected based on note context similarity. Finally, concatenative synthesis was used to automatically synthesize expressive performances of new pieces using the induced models. Supplemental online material for this article containing musical examples of the automatically generated ornamented pieces can be accessed at doi: 10.1080/17459737.2016.1207814 and https://soundcloud.com/machine-learning-and-jazz. In the Online Supplement we present an example of the musical piece Yesterdays by Jerome Kern, which was modeled using our methodology for expressive music performance in jazz guitar., This project has received funding from: the European Union Horizon 2020 research and innovation programme [grant agreement No 688269]; the Spanish TIN project TIMUL [grant agreement TIN2013-48152-C2-2-R].
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Evaluation of a sound quality visual feedback system for bow learning technique in violin beginners: an EEG study

  • Blanco, Angel David
  • Ramírez, Rafael, 1966-
Current music technologies can assist in the process of learning to play a musical instrument and provide objective measures for evaluating the improvement of music students in concrete music tasks. In this paper, we investigated the effects of a sound quality visual feedback system (SQVFS) in violin learning. In particular, we studied the EEG activity of a group of participants with no previous violin playing experience while they learned to produce a stable sound (regarding pitch, dynamics, and timbre) in order to find motor learning biomarkers in a music task. Eighteen subjects with no prior experience in violin playing were divided into two groups: participants in the first group (experimental group, N = 9) practiced with instructional videos and offline feedback from the SQVFS provided in alternation with their performance, while participants in a second group (control group, N = 9) practiced with the instructional videos only. A third group of violin experts (players with more than 6 years of experience) performed the same task for comparative purposes (N = 7). All participants were asked to perform 20 trials (4 blocks of 5 trials) consisting of a violin bowing exercise while their EEG activity and their produced sound was recorded. Significant sound quality improvements along the session were found in all participants with the exception of participants in the expert group. In addition, participants in the experimental group showed increased interest in the learning process and significant improvement after the second block not present in the control group. A significant correlation between the levels of frontal gamma band power and the sound improvement along the task was found in both the experimental and control group. This result is consistent with the temporal binding model which associates gamma band power with the role of integrating (binding) information processed in distributed cortical areas. Task complexity demands more cognitive resources, more binding and thus, gamma band power enhancement, which may be reduced as the demanded task begins to be automated as it is likely to be the case in both beginners groups., This work was partly sponsored by Fundación Memora, the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), and the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

JamSketch: improvisation support system with GA-based melody creation from user’s drawing

  • Kitahara, Tetsuro
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada al 13th International Symposium (CMMR), celebrat del 25 al 28 de setembre de 2017 a Matosinhos, Portugal., Improvisation is an enjoyable form of music performance but requires advanced skills and knowledge of music because the player has to create melodies immediately during the performance. To support improvisations by people without skills or knowledge of music, we have to develop (1) a human interface that can be used without skills or knowledge of music and (2) automatic melody generation from the user’s input that may be musically abstract or incomplete. In this paper, we develop an improvisation support system based on melodic outlines, which represent the overall contour of melodies, with a function of melody generation using a genetic algorithm (GA). Once the user draws a melodic outline on the piano-roll display with the mouse or touch screen, the system immediately generates a melody using a GA with a fitness function based on the similarity to the outline, an N-gram probability, and entropy. The generated melody is performed expressively based on expression parameters calculated with an machine learning approach. The results of listening tests for comparing human performances and the system’s performances suggest that generated melodies have quality similar to performances by non-expert human performers., This work was supported by JSPS KAKENHI Grant Numbers 26240025, 26280089, 16K16180, 16H01744, 16KT0136, and 17H00749 as well as by the Spanish TIMuL Project (TIN2013-48152-C2-2-R), and the TELMI Project of the Horizon 2020 Research and Innovation Programme (grant agreement No. 688269).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Applying deep learning techniques to estimate patterns of musical gesture

  • Dalmazzo, David
  • Waddell, George
  • Ramírez, Rafael, 1966-
Repetitive practice is one of the most important factors in improving the performance of motor skills. This paper focuses on the analysis and classification of forearm gestures in the context of violin playing. We recorded five experts and three students performing eight traditional classical violin bow-strokes: martelé, staccato, detaché, ricochet, legato, trémolo, collé, and col legno. To record inertial motion information, we utilized the Myo sensor, which reports a multidimensional time-series signal. We synchronized inertial motion recordings with audio data to extract the spatiotemporal dynamics of each gesture. Applying state-of-the-art deep neural networks, we implemented and compared different architectures where convolutional neural networks (CNN) models demonstrated recognition rates of 97.147%, 3DMultiHeaded_CNN models showed rates of 98.553%, and rates of 99.234% were demonstrated by CNN_LSTM models. The collected data (quaternion of the bowing arm of a violinist) contained sufficient information to distinguish the bowing techniques studied, and deep learning methods were capable of learning the movement patterns that distinguish these techniques. Each of the learning algorithms investigated (CNN, 3DMultiHeaded_CNN, and CNN_LSTM) produced high classification accuracies which supported the feasibility of training classifiers. The resulting classifiers may provide the foundation of a digital assistant to enhance musicians' time spent practicing alone, providing real-time feedback on the accuracy and consistency of their musical gestures in performance., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

EEG-Based Analysis of the Emotional Effect of Music Therapy on Palliative Care Cancer Patients

  • Ramírez, Rafael, 1966-
  • Planas Domingo, Josep
  • Escudé, Nuria
  • Mercadé, Jordi
  • Farriols, Cristina
Music is known to have the power to induce strong emotions. The present study assessed, based on Electroencephalography (EEG) data, the emotional response of terminally ill cancer patients to a music therapy intervention in a randomized controlled trial. A sample of 40 participants from the palliative care unit in the Hospital del Mar in Barcelona was randomly assigned to two groups of 20. The first group [experimental group (EG)] participated in a session of music therapy (MT), and the second group [control group (CG)] was provided with company. Based on our previous work on EEG-based emotion detection, instantaneous emotional indicators in the form of a coordinate in the arousal-valence plane were extracted from the participants' EEG data. The emotional indicators were analyzed in order to quantify (1) the overall emotional effect of MT on the patients compared to controls, and (2) the relative effect of the different MT techniques applied during each session. During each MT session, five conditions were considered: I (initial patient's state before MT starts), C1 (passive listening), C2 (active listening), R (relaxation), and F (final patient's state). EEG data analysis showed a significant increase in valence (p = 0.0004) and arousal (p = 0.003) between I and F in the EG. No significant changes were found in the CG. This results can be interpreted as a positive emotional effect of MT in advanced cancer patients. In addition, according to pre- and post-intervention questionnaire responses, participants in the EG also showed a significant decrease in tiredness, anxiety and breathing difficulties, as well as an increase in levels of well-being. No equivalent changes were observed in the CG., This work was partly sponsored by Fundación Memora, the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), and the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Machine learning and music generation

  • Iñesta, José M.
  • Conklin, Darrell
  • Ramírez, Rafael, 1966-
We guest editors acknowledge the financial support of the following funding sources: the Ministerio de Economía y Competitividad project TIMuL [No. TIN2013–48152–C2–1–R]; [No. TIN2013–48152–C2–2–R, supported by UE FEDER funds]; the project Lrn2Cre8, which is funded by the Future and Emerging Technologies (FET) programme within the Seventh Framework Programme for Research of the European Commission [FET grant number 610859]; the European Union Horizon 2020 research and innovation programme [grant agreement number 688269].
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Computational modelling of timbre dimensions for automatic violin tone quality assessment

  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
  • Waddell, George
  • Williamon, Aaron
Automatic assessment of music performance is an open research area widely studied in the past. A vast amount of systems aiming to enhance the learning process of a musical instrument are being developed in the recent years. However, most of the systems focus on the assessment of pitch and onset accuracy, and very few pay attention to tone quality. This is particularly true in violin music education, where although a consensus exist on what is a good or a bad tone quality, there is not a formal definition due to its subjectivity. We present a machine learning approach for the automatic assessment of violin tone quality. We depart from our previous work on the preliminary modelling of several dimensions involving tone quality. Based on recorded examples of tones with different qualities defined and recorded by a professional violinist, we applied machine learning techniques to learn computational models able to evaluate tone quality from extracted audio features. The tone quality models were implemented into a real-time-visual-feedback system., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

JamSketch: a drawing-based real-time evolutionary improvisation support system

  • Kitahara, Tetsuro
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada a: International Conference on New Interfaces for Musical Expression celebrat del 15 al 19 de maig de 2017 a Copenhagen, Dinamarca., In this paper, we present JamSketch, a real-time improvisation support system which automatically generates melodies according to melodic outlines drawn by the users. The system generates the improvised melodies based on (1) an outline sketched by the user using a mouse or a touch screen, (2) a genetic algorithm based on a dataset of existing music pieces as well as musical knowledge, and (3) an expressive performance model for timing and dynamic transformations. The aim of the system is to allow people with no prior musical knowledge to be able to enjoy playing music by improvising melodies in real time., This work was supported by JSPS KAKENHI Grant Numbers 26240025, 26280089, 16K16180, 16H01744, and 16KT0136, as well as by the Spanish TIMuL Project (TIN2013-48152-C2-2-R), and the TELMI Project of the Horizon 2020 Research and Innovation Programme (grant agreement No. 688269).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

A Machine learning approach to violin bow technique classification: a comparison between IMU and MOCAP systems

  • Dalmazzo, David
  • Tassani, Simone
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 5th international Workshop on Sensor-based Activity Recognition and Interaction celebrat el 20 i 21 de setembre de 2018 a Berlin, Alemanya., Motion Capture (MOCAP) Systems have been used to analyze body motion and postures in biomedicine, sports, rehabilitation, and music. With the aim to compare the precision of low-cost devices for motion tracking (e.g. Myo) with the precision of MOCAP systems in the context of music performance, we recorded MOCAP and Myo data of a top professional violinist executing four fundamental bowing techniques (i.e. Détaché, Martelé, Spiccato and Ricochet). Using the recorded data we applied machine learning techniques to train models to classify the four bowing techniques. Despite intrinsic differences between the MOCAP and low-cost data, the Myo-based classifier resulted in slightly higher accuracy than the MOCAP-based classifier. This result shows that it is possible to develop music-gesture learning applications based on low-cost technology which can be used in home environments for self-learning practitioners., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

The EyeHarp: A gaze-controlled digital musical instrument

  • Vamvakousis, Zacharias
  • Ramírez, Rafael, 1966-
We present and evaluate the EyeHarp, a new gaze-controlled Digital Musical Instrument, which aims to enable people with severe motor disabilities to learn, perform, and compose music using only their gaze as control mechanism. It consists of (1) a step-sequencer layer, which serves for constructing chords/arpeggios, and (2) a melody layer, for playing melodies and changing the chords/arpeggios. We have conducted a pilot evaluation of the EyeHarp involving 39 participants with no disabilities from both a performer and an audience perspective. In the first case, eight people with normal vision and no motor disability participated in a music-playing session in which both quantitative and qualitative data were collected. In the second case 31 people qualitatively evaluated the EyeHarp in a concert setting consisting of two parts: a solo performance part, and an ensemble (EyeHarp, two guitars, and flute) performance part. The obtained results indicate that, similarly to traditional music instruments, the proposed digital musical instrument has a steep learning curve, and allows to produce expressive performances both from the performer and audience perspective., This project has received funding from the European Unions Horizon 2020 research and innovation program under grant agreement No. 688269, as well as from the Spanish TIN project TIMUL under grant agreement TIN2013-48152-C2-2-R.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Mapping between dynamic markings and performed loudness: a machine learning approach

  • Kosta, Katerina
  • Ramírez, Rafael, 1966-
  • Bandtlow, Oscar F.
  • Chew, Elaine
Loudness variation is one of the foremost tools for expressivity in music performance. Loudness is frequently notated as dynamic markings such as p (piano, meaning soft) or f (forte, meaning loud). While dynamic markings in music scores are important indicators of how music pieces should be interpreted, their meaning is less straightforward than it may seem, and depends highly on the context in which they appear. In this article, we investigate the relationship between dynamic markings in the score and performed loudness by applying machine learning techniques – decision trees, support vector machines, artificial neural networks, and a k-nearest neighbor method – to the prediction of loudness levels corresponding to dynamic markings, and to the classification of dynamic markings given loudness values. The methods are applied to 44 recordings of performances of Chopin's Mazurkas, each by 8 pianists. The results show that loudness values and markings can be predicted relatively well when trained across recordings of the same piece, but fail dismally when trained across the pianist's recordings of other pieces, demonstrating that score features may trump individual style when modeling loudness choices. Evidence suggests that all the features chosen for the task are relevant, and analysis of the results reveals the forms (such as the return of the theme) and structures (such as dynamic-marking repetitions) that influence the predictability of loudness and markings. Modeling of loudness trends in expressive performance appears to be a delicate matter, and sometimes loudness expression can be a matter of the performer's idiosyncracy., This work was supported in part by a UK EPSRC Platform Grant for Digital Music [EP/K009559/1]; the Spanish TIN project TIMUL [TIN2013-48152-C2-2-R]; and the European Union's Horizon 2020 research and innovation programme [grant agreement No. 688269].
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Computational modelling of expressive music performance in hexaphonic guitar

  • Siquier, Marc
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 10th International Workshop on Machine Learning and Music (MML), celebrat a Barcelona (Espanya), el 6 d'octubre de 2017., Computational modelling of expressive music performance has been widely studied in the past. While previous work in this area has been mainly focused on classical piano music, there has been very little work on guitar music, and such work has focused on monophonic guitar playing. In this work, we present a machine learning approach to automatically generate expressive performances from non expressive music scores for polyphonic guitar. We treated guitar as an hexaphonic instrument, obtaining a polyphonic transcription of performed musical pieces. Features were extracted from the scores and performance actions were calculated from the deviations of the score and the performance. Machine learning techniques were used to train computational models to predict the aforementioned performance actions. Qualitative and quantitative evaluations of the models and the predicted pieces were performed., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), the European Union Horizon 2020 research and inno- vation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Bowing gestures classification in violin performance: a machine learning approach

  • Dalmazzo, David
  • Ramírez, Rafael, 1966-
Gestures in music are of paramount importance partly because they are directly linked to musicians' sound and expressiveness. At the same time, current motion capture technologies are capable of detecting body motion/gestures details very accurately. We present a machine learning approach to automatic violin bow gesture classification based on Hierarchical Hidden Markov Models (HHMM) and motion data. We recorded motion and audio data corresponding to seven representative bow techniques (Détaché, Martelé, Spiccato, Ricochet, Sautillé, Staccato, and Bariolage) performed by a professional violin player. We used the commercial Myo device for recording inertial motion information from the right forearm and synchronized it with audio recordings. Data was uploaded into an online public repository. After extracting features from both the motion and audio data, we trained an HHMM to identify the different bowing techniques automatically. Our model can determine the studied bowing techniques with over 94% accuracy. The results make feasible the application of this work in a practical learning scenario, where violin students can benefit from the real-time feedback provided by the system., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

A Machine learning approach to discover rules for expressive performance actions in jazz guitar music

  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Expert musicians introduce expression in their performances by manipulating sound properties such as timing, energy, pitch, and timbre. Here, we present a data driven computational approach to induce expressive performance rule models for note duration, onset, energy, and ornamentation transformations in jazz guitar music. We extract high-level features from a set of 16 commercial audio recordings (and corresponding music scores) of jazz guitarist Grant Green in order to characterize the expression in the pieces. We apply machine learning techniques to the resulting features to learn expressive performance rule models. We (1) quantitatively evaluate the accuracy of the induced models, (2) analyse the relative importance of the considered musical features, (3) discuss some of the learnt expressive performance rules in the context of previous work, and (4) assess their generailty. The accuracies of the induced predictive models is significantly above base-line levels indicating that the audio performances and the musical features extracted contain sufficient information to automatically learn informative expressive performance patterns. Feature analysis shows that the most important musical features for predicting expressive transformations are note duration, pitch, metrical strength, phrase position, Narmour structure, and tempo and key of the piece. Similarities and differences between the induced expressive rules and the rules reported in the literature were found. Differences may be due to the fact that most previously studied performance data has consisted of classical music recordings. Finally, the rules' performer specificity/generality is assessed by applying the induced rules to performances of the same pieces performed by two other professional jazz guitar players. Results show a consistency in the ornamentation patterns between Grant Green and the other two musicians, which may be interpreted as a good indicator for generality of the ornamentation rules., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), and the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Automatic assessment of violin performance using dynamic time warping classification

  • Giraldo, Sergio
  • Ortega, Ariadna
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Ramírez, Rafael, 1966-
  • Waddell, George
  • Williamon, Aaron
Comunicació presentada a: 26th Signal Processing and Communications Applications Conference, celebrada a Izmir, Turkey, del 2 al 5 de maig de 2018., The automatic assessment of music performance has become an area of special interest due to the increasing amount of technology-enhanced music learning systems. However, in most of these systems the assessment of the musical performance is based on the accuracy of onsets and pitch, paying little attention to other relevant aspects of performance. In this paper we present a preliminary study to assess the quality of violin performance using machine learning techniques. We collect recording examples of selected violin exercises varying from expert to amateur performances. We process the audio signal to extract features to train models using clustering based on Dynamic Time Warping distance. The quality of new performances is evaluated based on the level of match/miss-match to each of the recorded training examples., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Evaluation of audio-based feedback technologies for bow learning technique in violin beginners

  • Blanco, Angel David
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education (MIE'17), celebrat en Glasgow (Regne Unit) el 13 de novembre de 2017., We present a study of the effects of feedback technologies on the learning process of novice violin students. Twenty-one subjects participated in our experiment, divided into two groups: Beginners (participants with no prior violin playing experience, N=14), and experts (participants with more than 6 years of violin playing experience, N=7). The beginners group was further divided into two: a group of beginners learning with Youtube videos (N=7), and a group of beginners with additional feedback related to the quality of their performance (N=7). Participants were asked to perform a violin exercise during 21 trials while their audio was recorded and analyzed. Three different audio descriptors were extracted from each audio in order to evaluate the quality of the performance: Dynamic stability, pitch stability and aperiodicity. Beginners showed a significant improvement during the session(i.e. by comparing the beginning and the end of the session)in the quality of the sound recorded, while experts maintained their results. However, only the beginner group with feedback showed significant improvement between the middle and late part of the session, while the group without feedback remained stable., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Air violin: a machine learning approach to fingering gesture recognition

  • Dalmazzo, David
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education, celebrat el 13 de novembre de 2017 a Glasgow, Regne Unit., We train and evaluate two machine learning models for predicting fingering in violin performances using motion and EMG sensors integrated in the Myo device. Our aim is twofold: first, provide a fingering recognition model in the context of a gamification virtual violin application where we measure both right hand (i.e. bow) and left hand (i.e. fingering) gestures, and second, implement a tracking system for a computer assisted pedagogical tool for self-regulated learners in high-level music education. Our approach is based on the principle of mapping-by-demonstration in which the model is trained by the performer. We evaluated a model based on Decision Trees and compared it with a Hidden Markovian Model., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Hexaphonic guitar transcription and visualisation

  • Angulo, Iñigo
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada a: Second International Conference on Technologies for Music Notation and Representation (TENOR 2016) celebrat del 27 al 29 de maig de 2016 a Cambridge, Regne Unit., Music representation has been a widely researched topic through centuries. Transcription of music through the conventional notation system has dominated the field, for the best part of the last centuries. However, this notational system often falls short of communicating the essence of music to the masses, especially to the people with no music training. Advances in signal processing and computer science over the last few decades have bridged this gap to an extent, but conveying the meaning of music remains a challenging research field. Music visualisation is one such bridge, which we explore in this paper. This paper presents an approach to visualize guitar performances, transcribing musical events into visual forms. To achieve this, hexaphonic guitar processing is carried out (i.e. processing each of the six strings as an independent monophonic sound source) to get music descriptors, which reflect the most relevant features of a sound to define/characterise it. Once this information is obtained, our goal is to analyse how different mappings to the visual domain can meaningfully/intuitively represent music. As a final result, a system is proposed to enrich the musical listening experience, by extending the perceived auditory sensations to include visual stimuli., This work has been partly sponsored by the Spanish TIN TIMUL project (TIN2013- 48152-C2-2-R) and the H2020-ICT-688268 TELMI project.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Jazz ensemble expressive performance modeling

  • Bantula, Helena
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada a la 17th International Society for Music Information Retrieval Conference (ISMIR 2016), celebrada els dies 7 a 11 d'agost de 2016 a Nova York, EUA., Computational expressive music performance studies the analysis and characterisation of the deviations that a musician introduces when performing a musical piece. It has been studied in a classical context where timing and dynamic deviations are modeled using machine learning techniques. In jazz music, work has been done previously on the study of ornament prediction in guitar performance, as well as in saxophone expressive modeling. However, little work has been done on expressive ensemble performance. In this work, we analysed the musical expressivity of jazz guitar and piano from two different perspectives: solo and ensemble performance. The aim of this paper is to study the influence of piano accompaniment into the performance of a guitar melody and vice versa. Based on a set of recordings made by professional musicians, we extracted descriptors from the score, we transcribed the guitar and the piano performances and calculated performance actions for both instruments. We applied machine learning techniques to train models for each performance action, taking into account both solo and ensemble descriptors. Finally, we compared the accuracy of the induced models. The accuracy of most models increased when ensemble information was considered, which can be explained by the interaction between musicians., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R) and the H2020-ICT-688269 TELMI project.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Automatic assessment of tone quality in violin music performance

  • Giraldo, Sergio
  • Waddell, George
  • Nou Plana, Ignasi
  • Ortega, Ariadna
  • Mayor, Oscar
  • Perez, Alfonso
  • Williamon, Aaron
  • Ramírez, Rafael, 1966-
The automatic assessment of music performance has become an area of increasing interest due to the growing number of technology-enhanced music learning systems. In most of these systems, the assessment of musical performance is based on pitch and onset accuracy, but very few pay attention to other important aspects of performance, such as sound quality or timbre. This is particularly true in violin education, where the quality of timbre plays a significant role in the assessment of musical performances. However, obtaining quantifiable criteria for the assessment of timbre quality is challenging, as it relies on consensus among the subjective interpretations of experts. We present an approach to assess the quality of timbre in violin performances using machine learning techniques. We collected audio recordings of several tone qualities and performed perceptual tests to find correlations among different timbre dimensions. We processed the audio recordings to extract acoustic features for training tone-quality models. Correlations among the extracted features were analyzed and feature information for discriminating different timbre qualities were investigated. A real-time feedback system designed for pedagogical use was implemented in which users can train their own timbre models to assess and receive feedback on their performances., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Bowing modeling for violin students assistance

  • Ortega, Fabio J. M.
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education, que va tenir lloc el 13 de novembre de 2017 a Glasgow, UK., Though musicians tend to agree on the importance of practicing expressivity in performance, not many tools and techniques are available for the task. A machine learning model is proposed for predicting bowing velocity during performances of violin pieces. Our aim is to provide feedback to violin students in a technology–enhanced learning setting. Predictions are generated for musical phrases in a score by matching them to melodically and rhythmically similar phrases in performances by experts and adapting the bow velocity curve measured in the experts’ performance. Results show that mean error in velocity predictions and bowing direction classification accuracy outperform our baseline when reference phrases similar to the predicted ones are available., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Phrase-level modeling of expression in violin performances

  • Ortega, Fabio J. M.
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 10th International Workshop on Machine Learning and Music (MML), celebrat a Barcelona (Espanya), el 6 d'octubre de 2017., A model is proposed for predicting expressive variations in dynamics for violin performances with the purpose of facilitating expressive performance learning by students. The model uses phrases rather than single notes as units of analysis in a lazy learning approach: each phrase in a new score is matched to phrases from expressive performances by experts, adapting the experts’ transformations to render an expressive performance of the new score. In preliminary tests, the model approximates the dynamics of actual performances better than an unexpressive baseline model whenever the reference dataset contains melodies similar to those being predicted., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Real-time sound and motion feedback for violin bow technique learning: a controlled, randomized trial

  • Blanco, Angel David
  • Tassani, Simone
  • Ramírez, Rafael, 1966-
The production of good sound generation in the violin is a complex task that requires coordination and spatiotemporal control of bowing gestures. The use of motion-capture technologies to improve performance or reduce injury risks in the area of kinesiology is becoming widespread. The combination of motion accuracy and sound quality feedback has the potential of becoming an important aid in violin learning. In this study, we evaluate motion-capture and sound-quality analysis technologies developed inside the context of the TELMI, a technology-enhanced music learning project. We analyzed the sound and bow motion of 50 participants with no prior violin experience while learning to produce a stable sound in the violin. Participants were divided into two groups: the experimental group (N = 24) received real-time visual feedback both on kinematics and sound quality, while participants in the control group (N = 26) practiced without any type of external help. An additional third group of violin experts performed the same task for comparative purposes (N = 15). After the practice session, all groups were evaluated in a transfer phase without feedback. At the practice phase, the experimental group improved their bowing kinematics in comparison to the control group, but this was at the expense of impairing the sound quality of their performance. At the retention phase, the experimental group showed better results in sound quality, especially concerning control of sound dynamics. Besides, we found that the expert group improved the stability of their sound while using the technology. All in all, these results emphasize the importance of feedback technologies in learning complex tasks, such as musical instrument learning., This work was partly sponsored by Fundación Memora, the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), and the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Automatic assessment of tone quality in violin music performance

  • Giraldo, Sergio
  • Waddell, George
  • Nou Plana, Ignasi
  • Ortega, Ariadna
  • Mayor, Oscar
  • Perez, Alfonso
  • Williamon, Aaron
  • Ramírez, Rafael, 1966-
The automatic assessment of music performance has become an area of increasing interest due to the growing number of technology-enhanced music learning systems. In most of these systems, the assessment of musical performance is based on pitch and onset accuracy, but very few pay attention to other important aspects of performance, such as sound quality or timbre. This is particularly true in violin education, where the quality of timbre plays a significant role in the assessment of musical performances. However, obtaining quantifiable criteria for the assessment of timbre quality is challenging, as it relies on consensus among the subjective interpretations of experts. We present an approach to assess the quality of timbre in violin performances using machine learning techniques. We collected audio recordings of several tone qualities and performed perceptual tests to find correlations among different timbre dimensions. We processed the audio recordings to extract acoustic features for training tone-quality models. Correlations among the extracted features were analyzed and feature information for discriminating different timbre qualities were investigated. A real-time feedback system designed for pedagogical use was implemented in which users can train their own timbre models to assess and receive feedback on their performances., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Phrase-Level modeling of expression in violin performances

  • Ortega, Fabio J. M.
  • Giraldo, Sergio
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Ramírez, Rafael, 1966-
Background: Expression is a key skill in music performance, and one that is difficult to address in music lessons. Computational models that learn from expert performances can help providing suggestions and feedback to students. Aim: We propose and analyze an approach to modeling variations in dynamics and note onset timing for solo violin pieces with the purpose of facilitating expressive performance learning in new pieces, for which no reference performance is available. Method: The method generates phrase–level predictions based on musical score information on the assumption that expressiveness is idiomatic, and thus influenced by similar–sounding melodies. Predictions were evaluated numerically using three different datasets and against note–level machine–learning models, and also perceptually by listeners, who were presented to synthesized versions of musical excerpts, and asked to choose the most human–sounding one. Some of the presented excerpts were synthesized to reflect the variations in dynamics and timing predicted by the model, whereas others were shaped to reflect the dynamics and timing of an actual expert performance, and a third group was presented with no expressive variations. Results: surprisingly, none of the three synthesized versions was consistently selected as human–like nor preferred with statistical significance by listeners. Possible interpretations of these results include the fact that the melodies might have been impossible to interpret outside their musical context, or that expressive features that were left out of the modeling such as note articulation and vibrato are, in fact, essential to the perception of expression in violin performance. Positive feedback by some listeners toward the modeled melodies in a blind setting indicate that the modeling approach was capable of generating appropriate renditions at least for a subset of the data. Numerically, performance in phrase–level suffers a small degradation if compared to note–level, but produces predictions easier to interpret visually, thus more useful in a pedagogical setting., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Phrase-level modeling of expression in violin performances

  • Ortega, Fabio J. M.
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 10th International Workshop on Machine Learning and Music (MML), celebrat a Barcelona (Espanya), el 6 d'octubre de 2017., A model is proposed for predicting expressive variations in dynamics for violin performances with the purpose of facilitating expressive performance learning by students. The model uses phrases rather than single notes as units of analysis in a lazy learning approach: each phrase in a new score is matched to phrases from expressive performances by experts, adapting the experts’ transformations to render an expressive performance of the new score. In preliminary tests, the model approximates the dynamics of actual performances better than an unexpressive baseline model whenever the reference dataset contains melodies similar to those being predicted., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Evaluation of audio-based feedback technologies for bow learning technique in violin beginners

  • Blanco, Angel David
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education (MIE'17), celebrat en Glasgow (Regne Unit) el 13 de novembre de 2017., We present a study of the effects of feedback technologies on the learning process of novice violin students. Twenty-one subjects participated in our experiment, divided into two groups: Beginners (participants with no prior violin playing experience, N=14), and experts (participants with more than 6 years of violin playing experience, N=7). The beginners group was further divided into two: a group of beginners learning with Youtube videos (N=7), and a group of beginners with additional feedback related to the quality of their performance (N=7). Participants were asked to perform a violin exercise during 21 trials while their audio was recorded and analyzed. Three different audio descriptors were extracted from each audio in order to evaluate the quality of the performance: Dynamic stability, pitch stability and aperiodicity. Beginners showed a significant improvement during the session(i.e. by comparing the beginning and the end of the session)in the quality of the sound recorded, while experts maintained their results. However, only the beginner group with feedback showed significant improvement between the middle and late part of the session, while the group without feedback remained stable., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A Machine learning approach to ornamentation modeling and synthesis in jazz guitar

  • Ramírez, Rafael, 1966-
  • Giraldo, Sergio
We present a machine learning approach to automatically generate expressive (ornamented) jazz performances from un-expressive music scores. Features extracted from the scores and the corresponding audio recordings performed by a professional guitarist were used to train computational models for predicting melody ornamentation. As a first step, several machine learning techniques were explored to induce regression models for timing, onset, and dynamics (i.e. note duration and energy) transformations, and an ornamentation model for classifying notes as ornamented or non-ornamented. In a second step, the most suitable ornament for predicted ornamented notes was selected based on note context similarity. Finally, concatenative synthesis was used to automatically synthesize expressive performances of new pieces using the induced models. Supplemental online material for this article containing musical examples of the automatically generated ornamented pieces can be accessed at doi: 10.1080/17459737.2016.1207814 and https://soundcloud.com/machine-learning-and-jazz. In the Online Supplement we present an example of the musical piece Yesterdays by Jerome Kern, which was modeled using our methodology for expressive music performance in jazz guitar., This project has received funding from: the European Union Horizon 2020 research and innovation programme [grant agreement No 688269]; the Spanish TIN project TIMUL [grant agreement TIN2013-48152-C2-2-R].
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Computational modelling of timbre dimensions for automatic violin tone quality assessment

  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
  • Waddell, George
  • Williamon, Aaron
Automatic assessment of music performance is an open research area widely studied in the past. A vast amount of systems aiming to enhance the learning process of a musical instrument are being developed in the recent years. However, most of the systems focus on the assessment of pitch and onset accuracy, and very few pay attention to tone quality. This is particularly true in violin music education, where although a consensus exist on what is a good or a bad tone quality, there is not a formal definition due to its subjectivity. We present a machine learning approach for the automatic assessment of violin tone quality. We depart from our previous work on the preliminary modelling of several dimensions involving tone quality. Based on recorded examples of tones with different qualities defined and recorded by a professional violinist, we applied machine learning techniques to learn computational models able to evaluate tone quality from extracted audio features. The tone quality models were implemented into a real-time-visual-feedback system., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Automatic assessment of violin performance using dynamic time warping classification

  • Giraldo, Sergio
  • Ortega, Ariadna
  • Pérez Carrillo, Alfonso Antonio, 1977-
  • Ramírez, Rafael, 1966-
  • Waddell, George
  • Williamon, Aaron
Comunicació presentada a: 26th Signal Processing and Communications Applications Conference, celebrada a Izmir, Turkey, del 2 al 5 de maig de 2018., The automatic assessment of music performance has become an area of special interest due to the increasing amount of technology-enhanced music learning systems. However, in most of these systems the assessment of the musical performance is based on the accuracy of onsets and pitch, paying little attention to other relevant aspects of performance. In this paper we present a preliminary study to assess the quality of violin performance using machine learning techniques. We collect recording examples of selected violin exercises varying from expert to amateur performances. We process the audio signal to extract features to train models using clustering based on Dynamic Time Warping distance. The quality of new performances is evaluated based on the level of match/miss-match to each of the recorded training examples., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Air violin: a machine learning approach to fingering gesture recognition

  • Dalmazzo, David
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education, celebrat el 13 de novembre de 2017 a Glasgow, Regne Unit., We train and evaluate two machine learning models for predicting fingering in violin performances using motion and EMG sensors integrated in the Myo device. Our aim is twofold: first, provide a fingering recognition model in the context of a gamification virtual violin application where we measure both right hand (i.e. bow) and left hand (i.e. fingering) gestures, and second, implement a tracking system for a computer assisted pedagogical tool for self-regulated learners in high-level music education. Our approach is based on the principle of mapping-by-demonstration in which the model is trained by the performer. We evaluated a model based on Decision Trees and compared it with a Hidden Markovian Model., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

JamSketch: a drawing-based real-time evolutionary improvisation support system

  • Kitahara, Tetsuro
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada a: International Conference on New Interfaces for Musical Expression celebrat del 15 al 19 de maig de 2017 a Copenhagen, Dinamarca., In this paper, we present JamSketch, a real-time improvisation support system which automatically generates melodies according to melodic outlines drawn by the users. The system generates the improvised melodies based on (1) an outline sketched by the user using a mouse or a touch screen, (2) a genetic algorithm based on a dataset of existing music pieces as well as musical knowledge, and (3) an expressive performance model for timing and dynamic transformations. The aim of the system is to allow people with no prior musical knowledge to be able to enjoy playing music by improvising melodies in real time., This work was supported by JSPS KAKENHI Grant Numbers 26240025, 26280089, 16K16180, 16H01744, and 16KT0136, as well as by the Spanish TIMuL Project (TIN2013-48152-C2-2-R), and the TELMI Project of the Horizon 2020 Research and Innovation Programme (grant agreement No. 688269).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Computational modelling of expressive music performance in hexaphonic guitar

  • Siquier, Marc
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 10th International Workshop on Machine Learning and Music (MML), celebrat a Barcelona (Espanya), el 6 d'octubre de 2017., Computational modelling of expressive music performance has been widely studied in the past. While previous work in this area has been mainly focused on classical piano music, there has been very little work on guitar music, and such work has focused on monophonic guitar playing. In this work, we present a machine learning approach to automatically generate expressive performances from non expressive music scores for polyphonic guitar. We treated guitar as an hexaphonic instrument, obtaining a polyphonic transcription of performed musical pieces. Features were extracted from the scores and performance actions were calculated from the deviations of the score and the performance. Machine learning techniques were used to train computational models to predict the aforementioned performance actions. Qualitative and quantitative evaluations of the models and the predicted pieces were performed., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), the European Union Horizon 2020 research and inno- vation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Evaluation of a sound quality visual feedback system for bow learning technique in violin beginners: an EEG study

  • Blanco, Angel David
  • Ramírez, Rafael, 1966-
Current music technologies can assist in the process of learning to play a musical instrument and provide objective measures for evaluating the improvement of music students in concrete music tasks. In this paper, we investigated the effects of a sound quality visual feedback system (SQVFS) in violin learning. In particular, we studied the EEG activity of a group of participants with no previous violin playing experience while they learned to produce a stable sound (regarding pitch, dynamics, and timbre) in order to find motor learning biomarkers in a music task. Eighteen subjects with no prior experience in violin playing were divided into two groups: participants in the first group (experimental group, N = 9) practiced with instructional videos and offline feedback from the SQVFS provided in alternation with their performance, while participants in a second group (control group, N = 9) practiced with the instructional videos only. A third group of violin experts (players with more than 6 years of experience) performed the same task for comparative purposes (N = 7). All participants were asked to perform 20 trials (4 blocks of 5 trials) consisting of a violin bowing exercise while their EEG activity and their produced sound was recorded. Significant sound quality improvements along the session were found in all participants with the exception of participants in the expert group. In addition, participants in the experimental group showed increased interest in the learning process and significant improvement after the second block not present in the control group. A significant correlation between the levels of frontal gamma band power and the sound improvement along the task was found in both the experimental and control group. This result is consistent with the temporal binding model which associates gamma band power with the role of integrating (binding) information processed in distributed cortical areas. Task complexity demands more cognitive resources, more binding and thus, gamma band power enhancement, which may be reduced as the demanded task begins to be automated as it is likely to be the case in both beginners groups., This work was partly sponsored by Fundación Memora, the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), and the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Jazz ensemble expressive performance modeling

  • Bantula, Helena
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada a la 17th International Society for Music Information Retrieval Conference (ISMIR 2016), celebrada els dies 7 a 11 d'agost de 2016 a Nova York, EUA., Computational expressive music performance studies the analysis and characterisation of the deviations that a musician introduces when performing a musical piece. It has been studied in a classical context where timing and dynamic deviations are modeled using machine learning techniques. In jazz music, work has been done previously on the study of ornament prediction in guitar performance, as well as in saxophone expressive modeling. However, little work has been done on expressive ensemble performance. In this work, we analysed the musical expressivity of jazz guitar and piano from two different perspectives: solo and ensemble performance. The aim of this paper is to study the influence of piano accompaniment into the performance of a guitar melody and vice versa. Based on a set of recordings made by professional musicians, we extracted descriptors from the score, we transcribed the guitar and the piano performances and calculated performance actions for both instruments. We applied machine learning techniques to train models for each performance action, taking into account both solo and ensemble descriptors. Finally, we compared the accuracy of the induced models. The accuracy of most models increased when ensemble information was considered, which can be explained by the interaction between musicians., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R) and the H2020-ICT-688269 TELMI project.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Applying deep learning techniques to estimate patterns of musical gesture

  • Dalmazzo, David
  • Waddell, George
  • Ramírez, Rafael, 1966-
Repetitive practice is one of the most important factors in improving the performance of motor skills. This paper focuses on the analysis and classification of forearm gestures in the context of violin playing. We recorded five experts and three students performing eight traditional classical violin bow-strokes: martelé, staccato, detaché, ricochet, legato, trémolo, collé, and col legno. To record inertial motion information, we utilized the Myo sensor, which reports a multidimensional time-series signal. We synchronized inertial motion recordings with audio data to extract the spatiotemporal dynamics of each gesture. Applying state-of-the-art deep neural networks, we implemented and compared different architectures where convolutional neural networks (CNN) models demonstrated recognition rates of 97.147%, 3DMultiHeaded_CNN models showed rates of 98.553%, and rates of 99.234% were demonstrated by CNN_LSTM models. The collected data (quaternion of the bowing arm of a violinist) contained sufficient information to distinguish the bowing techniques studied, and deep learning methods were capable of learning the movement patterns that distinguish these techniques. Each of the learning algorithms investigated (CNN, 3DMultiHeaded_CNN, and CNN_LSTM) produced high classification accuracies which supported the feasibility of training classifiers. The resulting classifiers may provide the foundation of a digital assistant to enhance musicians' time spent practicing alone, providing real-time feedback on the accuracy and consistency of their musical gestures in performance., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Hexaphonic guitar transcription and visualisation

  • Angulo, Iñigo
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada a: Second International Conference on Technologies for Music Notation and Representation (TENOR 2016) celebrat del 27 al 29 de maig de 2016 a Cambridge, Regne Unit., Music representation has been a widely researched topic through centuries. Transcription of music through the conventional notation system has dominated the field, for the best part of the last centuries. However, this notational system often falls short of communicating the essence of music to the masses, especially to the people with no music training. Advances in signal processing and computer science over the last few decades have bridged this gap to an extent, but conveying the meaning of music remains a challenging research field. Music visualisation is one such bridge, which we explore in this paper. This paper presents an approach to visualize guitar performances, transcribing musical events into visual forms. To achieve this, hexaphonic guitar processing is carried out (i.e. processing each of the six strings as an independent monophonic sound source) to get music descriptors, which reflect the most relevant features of a sound to define/characterise it. Once this information is obtained, our goal is to analyse how different mappings to the visual domain can meaningfully/intuitively represent music. As a final result, a system is proposed to enrich the musical listening experience, by extending the perceived auditory sensations to include visual stimuli., This work has been partly sponsored by the Spanish TIN TIMUL project (TIN2013- 48152-C2-2-R) and the H2020-ICT-688268 TELMI project.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Bowing modeling for violin students assistance

  • Ortega, Fabio J. M.
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education, que va tenir lloc el 13 de novembre de 2017 a Glasgow, UK., Though musicians tend to agree on the importance of practicing expressivity in performance, not many tools and techniques are available for the task. A machine learning model is proposed for predicting bowing velocity during performances of violin pieces. Our aim is to provide feedback to violin students in a technology–enhanced learning setting. Predictions are generated for musical phrases in a score by matching them to melodically and rhythmically similar phrases in performances by experts and adapting the bow velocity curve measured in the experts’ performance. Results show that mean error in velocity predictions and bowing direction classification accuracy outperform our baseline when reference phrases similar to the predicted ones are available., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Machine learning and music generation

  • Iñesta, José M.
  • Conklin, Darrell
  • Ramírez, Rafael, 1966-
We guest editors acknowledge the financial support of the following funding sources: the Ministerio de Economía y Competitividad project TIMuL [No. TIN2013–48152–C2–1–R]; [No. TIN2013–48152–C2–2–R, supported by UE FEDER funds]; the project Lrn2Cre8, which is funded by the Future and Emerging Technologies (FET) programme within the Seventh Framework Programme for Research of the European Commission [FET grant number 610859]; the European Union Horizon 2020 research and innovation programme [grant agreement number 688269].
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Mapping between dynamic markings and performed loudness: a machine learning approach

  • Kosta, Katerina
  • Ramírez, Rafael, 1966-
  • Bandtlow, Oscar F.
  • Chew, Elaine
Loudness variation is one of the foremost tools for expressivity in music performance. Loudness is frequently notated as dynamic markings such as p (piano, meaning soft) or f (forte, meaning loud). While dynamic markings in music scores are important indicators of how music pieces should be interpreted, their meaning is less straightforward than it may seem, and depends highly on the context in which they appear. In this article, we investigate the relationship between dynamic markings in the score and performed loudness by applying machine learning techniques – decision trees, support vector machines, artificial neural networks, and a k-nearest neighbor method – to the prediction of loudness levels corresponding to dynamic markings, and to the classification of dynamic markings given loudness values. The methods are applied to 44 recordings of performances of Chopin's Mazurkas, each by 8 pianists. The results show that loudness values and markings can be predicted relatively well when trained across recordings of the same piece, but fail dismally when trained across the pianist's recordings of other pieces, demonstrating that score features may trump individual style when modeling loudness choices. Evidence suggests that all the features chosen for the task are relevant, and analysis of the results reveals the forms (such as the return of the theme) and structures (such as dynamic-marking repetitions) that influence the predictability of loudness and markings. Modeling of loudness trends in expressive performance appears to be a delicate matter, and sometimes loudness expression can be a matter of the performer's idiosyncracy., This work was supported in part by a UK EPSRC Platform Grant for Digital Music [EP/K009559/1]; the Spanish TIN project TIMUL [TIN2013-48152-C2-2-R]; and the European Union's Horizon 2020 research and innovation programme [grant agreement No. 688269].
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Bowing gestures classification in violin performance: a machine learning approach

  • Dalmazzo, David
  • Ramírez, Rafael, 1966-
Gestures in music are of paramount importance partly because they are directly linked to musicians' sound and expressiveness. At the same time, current motion capture technologies are capable of detecting body motion/gestures details very accurately. We present a machine learning approach to automatic violin bow gesture classification based on Hierarchical Hidden Markov Models (HHMM) and motion data. We recorded motion and audio data corresponding to seven representative bow techniques (Détaché, Martelé, Spiccato, Ricochet, Sautillé, Staccato, and Bariolage) performed by a professional violin player. We used the commercial Myo device for recording inertial motion information from the right forearm and synchronized it with audio recordings. Data was uploaded into an online public repository. After extracting features from both the motion and audio data, we trained an HHMM to identify the different bowing techniques automatically. Our model can determine the studied bowing techniques with over 94% accuracy. The results make feasible the application of this work in a practical learning scenario, where violin students can benefit from the real-time feedback provided by the system., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Acquisition of violin instrumental gestures using an infrared depth camera

  • Vamvakousis, Zacharias
  • Perez, Alfonso
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 15th Sound and Music Computing Conference (SMC2018), celebrat del 4 al 7 de juliol a Limassol, Xipre., We present a low-cost six-degrees of freedom violin and bow pose-tracking system. The infrared and depth streams of an infrared-based Microsoft Kinect Mbox One depth camera are utilized to track the 3-dimensional position of 6 infrared markers attached on the violin and the bow. After computing the bow pose on the violin coordinate system, a number of bowing features are extracted. In order to evaluate the system’s performance, we recorded 4 bowing exercises using simultaneously our system along with a commercial two-sensor 3D tracking system based on electromagnetic field (EMF) sensing. The mean pearson coefficient values were 0.996 for the bow position, 0.889 for the bow velocity, 0.966 for the bow tilt, 0.692 for the bowbridge distance and 0.998 for the bow inclination. Compared to existing bowing-tracking solutions, the proposed solution might be appealing because of its low cost, easy setup and high performance., This work was partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), and the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Musical neurofeedback for treating depression in elderly people

  • Ramírez, Rafael, 1966-
  • Palencia-Lefler Ors, Manuel
  • Giraldo, Sergio
  • Vamvakousis, Zacharias
We introduce a new neurofeedback approach, which allows users to manipulate expressive parameters in music performances using their emotional state, and we present the results of a pilot clinical experiment applying the approach to alleviate depression in elderly people. Ten adults (9 female and 1 male, mean = 84, SD = 5.8) with normal hearing participated in the neurofeedback study consisting of 10 sessions (2 sessions per week) of 15 min each. EEG data was acquired using the Emotiv EPOC EEG device. In all sessions, subjects were asked to sit in a comfortable chair facing two loudspeakers, to close their eyes, and to avoid moving during the experiment. Participants listened to music pieces preselected according to their music preferences, and were encouraged to increase the loudness and tempo of the pieces, based on their arousal and valence levels. The neurofeedback system was tuned so that increased arousal, computed as beta to alpha activity ratio in the frontal cortex corresponded to increased loudness, and increased valence, computed as relative frontal alpha activity in the right lobe compared to the left lobe, corresponded to increased tempo. Pre and post evaluation of six participants was performed using the BDI depression test, showing an average improvement of 17.2% (1.3) in their BDI scores at the end of the study. In addition, an analysis of the collected EEG data of the participants showed a significant decrease of relative alpha activity in their left frontal lobe (p = 0.00008), which may be interpreted as an improvement of their depression condition., This work has been partly sponsored by the Ministerio de Economia y Competitividad under Grant TIN2013-48152-C2-2-R (TIMuL Project).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A Machine learning approach to discover rules for expressive performance actions in jazz guitar music

  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Expert musicians introduce expression in their performances by manipulating sound properties such as timing, energy, pitch, and timbre. Here, we present a data driven computational approach to induce expressive performance rule models for note duration, onset, energy, and ornamentation transformations in jazz guitar music. We extract high-level features from a set of 16 commercial audio recordings (and corresponding music scores) of jazz guitarist Grant Green in order to characterize the expression in the pieces. We apply machine learning techniques to the resulting features to learn expressive performance rule models. We (1) quantitatively evaluate the accuracy of the induced models, (2) analyse the relative importance of the considered musical features, (3) discuss some of the learnt expressive performance rules in the context of previous work, and (4) assess their generailty. The accuracies of the induced predictive models is significantly above base-line levels indicating that the audio performances and the musical features extracted contain sufficient information to automatically learn informative expressive performance patterns. Feature analysis shows that the most important musical features for predicting expressive transformations are note duration, pitch, metrical strength, phrase position, Narmour structure, and tempo and key of the piece. Similarities and differences between the induced expressive rules and the rules reported in the literature were found. Differences may be due to the fact that most previously studied performance data has consisted of classical music recordings. Finally, the rules' performer specificity/generality is assessed by applying the induced rules to performances of the same pieces performed by two other professional jazz guitar players. Results show a consistency in the ornamentation patterns between Grant Green and the other two musicians, which may be interpreted as a good indicator for generality of the ornamentation rules., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), and the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Real-time sound and motion feedback for violin bow technique learning: a controlled, randomized trial

  • Blanco, Angel David
  • Tassani, Simone
  • Ramírez, Rafael, 1966-
The production of good sound generation in the violin is a complex task that requires coordination and spatiotemporal control of bowing gestures. The use of motion-capture technologies to improve performance or reduce injury risks in the area of kinesiology is becoming widespread. The combination of motion accuracy and sound quality feedback has the potential of becoming an important aid in violin learning. In this study, we evaluate motion-capture and sound-quality analysis technologies developed inside the context of the TELMI, a technology-enhanced music learning project. We analyzed the sound and bow motion of 50 participants with no prior violin experience while learning to produce a stable sound in the violin. Participants were divided into two groups: the experimental group (N = 24) received real-time visual feedback both on kinematics and sound quality, while participants in the control group (N = 26) practiced without any type of external help. An additional third group of violin experts performed the same task for comparative purposes (N = 15). After the practice session, all groups were evaluated in a transfer phase without feedback. At the practice phase, the experimental group improved their bowing kinematics in comparison to the control group, but this was at the expense of impairing the sound quality of their performance. At the retention phase, the experimental group showed better results in sound quality, especially concerning control of sound dynamics. Besides, we found that the expert group improved the stability of their sound while using the technology. All in all, these results emphasize the importance of feedback technologies in learning complex tasks, such as musical instrument learning., This work was partly sponsored by Fundación Memora, the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), and the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

The EyeHarp: A gaze-controlled digital musical instrument

  • Vamvakousis, Zacharias
  • Ramírez, Rafael, 1966-
We present and evaluate the EyeHarp, a new gaze-controlled Digital Musical Instrument, which aims to enable people with severe motor disabilities to learn, perform, and compose music using only their gaze as control mechanism. It consists of (1) a step-sequencer layer, which serves for constructing chords/arpeggios, and (2) a melody layer, for playing melodies and changing the chords/arpeggios. We have conducted a pilot evaluation of the EyeHarp involving 39 participants with no disabilities from both a performer and an audience perspective. In the first case, eight people with normal vision and no motor disability participated in a music-playing session in which both quantitative and qualitative data were collected. In the second case 31 people qualitatively evaluated the EyeHarp in a concert setting consisting of two parts: a solo performance part, and an ensemble (EyeHarp, two guitars, and flute) performance part. The obtained results indicate that, similarly to traditional music instruments, the proposed digital musical instrument has a steep learning curve, and allows to produce expressive performances both from the performer and audience perspective., This project has received funding from the European Unions Horizon 2020 research and innovation program under grant agreement No. 688269, as well as from the Spanish TIN project TIMUL under grant agreement TIN2013-48152-C2-2-R.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

EEG-Based Analysis of the Emotional Effect of Music Therapy on Palliative Care Cancer Patients

  • Ramírez, Rafael, 1966-
  • Planas Domingo, Josep
  • Escudé, Nuria
  • Mercadé, Jordi
  • Farriols, Cristina
Music is known to have the power to induce strong emotions. The present study assessed, based on Electroencephalography (EEG) data, the emotional response of terminally ill cancer patients to a music therapy intervention in a randomized controlled trial. A sample of 40 participants from the palliative care unit in the Hospital del Mar in Barcelona was randomly assigned to two groups of 20. The first group [experimental group (EG)] participated in a session of music therapy (MT), and the second group [control group (CG)] was provided with company. Based on our previous work on EEG-based emotion detection, instantaneous emotional indicators in the form of a coordinate in the arousal-valence plane were extracted from the participants' EEG data. The emotional indicators were analyzed in order to quantify (1) the overall emotional effect of MT on the patients compared to controls, and (2) the relative effect of the different MT techniques applied during each session. During each MT session, five conditions were considered: I (initial patient's state before MT starts), C1 (passive listening), C2 (active listening), R (relaxation), and F (final patient's state). EEG data analysis showed a significant increase in valence (p = 0.0004) and arousal (p = 0.003) between I and F in the EG. No significant changes were found in the CG. This results can be interpreted as a positive emotional effect of MT in advanced cancer patients. In addition, according to pre- and post-intervention questionnaire responses, participants in the EG also showed a significant decrease in tiredness, anxiety and breathing difficulties, as well as an increase in levels of well-being. No equivalent changes were observed in the CG., This work was partly sponsored by Fundación Memora, the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), and the European Union Horizon 2020 research and innovation program under grant agreement No. 688269 (TELMI project).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

A Real-time feedback learning tool to visualize sound quality in violin performances

  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
  • Waddell, George
  • Williamon, Aaron
Comunicació presentada a: 10th International Workshop on Machine Learning and Music (MML), celebrat a Barcelona (Espanya), el 6 d'octubre de 2017., The assessment of the sound properties of a performed mu- sical note has been widely studied in the past. Although a consensus exist on what is a good or a bad musical performance, there is not a formal de nition of performance tone quality due to its subjectivity. In this study we present a computational approach for the automatic assess- ment of violin sound production. We investigate the correlations among extracted features from audio performances and the perceptual quality of violin sounds rated by listeners using machine learning techniques. The obtained models are used for implementing a real-time feedback learning system., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN2013-48152-C2-2-R), the European Union Horizon 2020 research and inno- vation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

A Machine learning approach to violin bow technique classification: a comparison between IMU and MOCAP systems

  • Dalmazzo, David
  • Tassani, Simone
  • Ramírez, Rafael, 1966-
Comunicació presentada a: 5th international Workshop on Sensor-based Activity Recognition and Interaction celebrat el 20 i 21 de setembre de 2018 a Berlin, Alemanya., Motion Capture (MOCAP) Systems have been used to analyze body motion and postures in biomedicine, sports, rehabilitation, and music. With the aim to compare the precision of low-cost devices for motion tracking (e.g. Myo) with the precision of MOCAP systems in the context of music performance, we recorded MOCAP and Myo data of a top professional violinist executing four fundamental bowing techniques (i.e. Détaché, Martelé, Spiccato and Ricochet). Using the recorded data we applied machine learning techniques to train models to classify the four bowing techniques. Despite intrinsic differences between the MOCAP and low-cost data, the Myo-based classifier resulted in slightly higher accuracy than the MOCAP-based classifier. This result shows that it is possible to develop music-gesture learning applications based on low-cost technology which can be used in home environments for self-learning practitioners., This work has been partly sponsored by the Spanish TIN project TIMUL (TIN 2013-48152-C2-2-R), the European Union Horizon 2020 research and innovation programme under grant agreement No. 688269 (TELMI project), and the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

JamSketch: improvisation support system with GA-based melody creation from user’s drawing

  • Kitahara, Tetsuro
  • Giraldo, Sergio
  • Ramírez, Rafael, 1966-
Comunicació presentada al 13th International Symposium (CMMR), celebrat del 25 al 28 de setembre de 2017 a Matosinhos, Portugal., Improvisation is an enjoyable form of music performance but requires advanced skills and knowledge of music because the player has to create melodies immediately during the performance. To support improvisations by people without skills or knowledge of music, we have to develop (1) a human interface that can be used without skills or knowledge of music and (2) automatic melody generation from the user’s input that may be musically abstract or incomplete. In this paper, we develop an improvisation support system based on melodic outlines, which represent the overall contour of melodies, with a function of melody generation using a genetic algorithm (GA). Once the user draws a melodic outline on the piano-roll display with the mouse or touch screen, the system immediately generates a melody using a GA with a fitness function based on the similarity to the outline, an N-gram probability, and entropy. The generated melody is performed expressively based on expression parameters calculated with an machine learning approach. The results of listening tests for comparing human performances and the system’s performances suggest that generated melodies have quality similar to performances by non-expert human performers., This work was supported by JSPS KAKENHI Grant Numbers 26240025, 26280089, 16K16180, 16H01744, 16KT0136, and 17H00749 as well as by the Spanish TIMuL Project (TIN2013-48152-C2-2-R), and the TELMI Project of the Horizon 2020 Research and Innovation Programme (grant agreement No. 688269).