Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 13
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Analysis of musical dynamics in vocal performances using loudness measures

  • Narang, Jyoti
  • Miron, Marius
  • Srinivasamurthy, Ajay
  • Serra, Xavier
Comunicació presentada a: DAFx20in22, celebrat del 6 al 10 de setembre de 2022, a Viena, Austria., In addition to tone, pitch and rhythm, dynamics is one of the expressive dimensions of the performance of a music piece that has received limited attention. While the usage of dynamics may vary from artist to artist, and also from performance to performance, a systematic methodology to automatically identify the dynamics of a performance in terms of musically meaningful terms like forte, piano may offer valuable feedback in the context of music education and in particular in singing. To this end, we have manually annotated the dynamic markings of commercial recordings of popular rock and pop songs from the Smule Vocal Balanced (SVB) dataset which will be used as reference data. Then as a first step for our research goal, we propose a method to derive and compare singing voice loudness curves in polyphonic mixtures. Towards measuring the similarity and variation of dynamics, we compare the dynamics curves of the SVB renditions with the one derived from the original songs. We perform the same comparison using professionally produced renditions from a karaoke website. We relate high values of Spearman correlation coefficient found in some select student renditions and the professional renditions with accurate dynamics., Part of this research is funded by Project Musical AI - PID2019-111403GBI00/ AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Automatic piano fingering from partially annotated scores using autoregressive neural networks

  • Ramoneda, Pedro
  • Jeong, Dasaem
  • Nakamura, Eita
  • Serra, Xavier
  • Miron, Marius
Comunicació presentada a: 30th ACM International Conference on Multimedia (MM'22), celebrat del 10 al 14 d'octubre de 2022 a Lisboa, Portugal, Piano fingering is a creative and highly individualised task acquired by musicians progressively in their first music education years. Pianists must learn to choose the order of fingers to play the piano keys because scores do not have engraved finger and hand movements as other technique elements. Numerous research efforts have been conducted for automatic piano fingering based on a previous dataset composed of 150 score excerpts fully annotated by multiple expert annotators. However, most piano sheets include partial annotations for problematic finger and hand movements. We introduce a novel dataset for the task, the ThumbSet dataset, containing 2523 pieces with partial and noisy annotations of piano fingering crowdsourced from non-expert annotators. As part of our methodology, we propose two autoregressive neural networks with beam search decoding for modelling automatic piano fingering as a sequence-to-sequence learning problem, considering the correlation between output finger labels. We design the first model with the exact pitch representation of previous proposals. The second model uses graph neural networks to more effectively represent polyphony, whose treatment has been a common issue across previous studies. Finally, we finetune the models on the existing expert annotations dataset. The evaluation shows that (1) we are able to achieve high performance when training on the ThumbSet dataset and that (2) the proposed models outperform the state-of-the-art hidden Markov models and recurrent neural network baselines. Code, dataset, models, and results are made available to enhance the task reproducibility, including a new framework for evaluation, This work is supported in part by the project Musical AI - PID2019- 111403GB-I00/AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovacion y Universidades (MCIU) and the Agencia Estatal de Investigacion (AEI), Sogang University Research Grant of 202110035.01 and JSPS KAKENHI Nos. 21K12187 and 22H03661.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Contribución de congreso (ConferenceOutput)

From Simultaneous to Streaming Machine Translation by Leveraging Streaming History

  • Iranzo-Sánchez, Javier
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Juan, Alfons|||0000-0002-9984-4072
[EN] Simultaneous Machine Translation is the task of incrementally translating an input sentence before it is fully available. Currently, simultaneous translation is carried out by translating each sentence independently of the previously translated text. More generally, Streaming MT can be understood as an extension of Simultaneous MT to the incremental translation of a continuous input text stream. In this work, a state-of-the-art simultaneous sentencelevel MT system is extended to the streaming setup by leveraging the streaming history. Extensive empirical results are reported on IWSLT Translation Tasks, showing that leveraging the streaming history leads to significant quality gains. In particular, the proposed system proves to compare favorably to the best performing systems., The research leading to these results has received funding from the European Union¿s Horizon 2020 research and innovation programme under grant agreements no. 761758 (X5Gon) and 952215 (TAILOR), and Erasmus+ Education programme under grant agreement no. 20-226-093604-SCH (EXPERT); the Government of Spain¿s grant RTI2018-094879-B-I00 (Multisub) funded by MCIN/AEI/10.13039/501100011033 & ¿ERDF A way of making Europe¿, and FPU scholarships FPU18/04135; and the Generalitat Valenciana¿s research project Classroom Activity Recognition (ref. PROMETEO/2019/111). The authors gratefully acknowledge the computer resources at Artemisa, funded by the European Union ERDF and Comunitat Valenciana as well as the technical support provided by the Instituto de Física Corpuscular, IFIC (CSIC-UV).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Contribución de congreso (ConferenceOutput)

Transcranial acoustic holograms for arbitrary fields generation using focused ultrasound into the brain

  • Jiménez-Gambín, Sergio
  • Jimenez, Noe|||0000-0002-6539-670X
  • Benlloch Baviera, Jose María|||0000-0001-6073-1436
  • Camarena Femenia, Francisco|||0000-0002-6713-1414
[EN] We present 3D printed holographic lenses that correct the aberrations of the skull and, simultaneously, produce arbitrary ultrasonic fields with the geometry of brain structures. Using experimental techniques on a human skull phantom (HSP), a multiple-point focusing lens is designed to focus at both human hippocampi at once; a beam following an arbitrary curved trajectory, i.e., a self-bending beam; and a holographic plate producing a broad focus that overlaps with the left hippocampus (LH). Skull and LH geometries and acoustic properties are obtained from CT-scans and MRI, respectively. Time-reversal (TR) method is used to obtain the magnitude and phase of the back-propagated field from the target shape towards the lens surface. The holographic lenses are designed assuming each pixel of the lens vibrates as a Fabry-Pérot resonator. The resulting lenses are 3D printed using SLA techniques. The three studied cases show similar results in simulation and experiment with and without the HSP: for the bi-focal beam, the reconstructed field accurately matches the target foci; for the curved trajectory beam, the target acoustic image is reconstructed by the designed holographic lens; for the broad focus beam, results present the same qualitative performance providing a similar overall covering of the LH. The reported holographic lenses can be used to control the spatial features of ultrasonic beams inside the skull in an unprecedented manner using single-element ultrasonic sources., This work was supported by Generalitat Valenciana through grants APOSTD/2017/042, ACIF/2017/045 and GV/2018/11. FC acknowledges financial support from Agencia Valenciana de la Innovacio through grant INNCON00/18/9 and European Regional Development Fund (IDIFEDER/2018/022).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Contribución de congreso (ConferenceOutput)

Developing experimental design and analysis skills in Biomedical Engineering. A laboratory experiment proposal

  • Del-Canto, Irene
  • Morales-Tatay, José Manuel
  • Moratal, David|||0000-0002-2825-3646
[EN] Learning-teaching process in the area of Biomedical Engineering requires the acquisition of skills related to the formulation and resolution of problems in biology and medicine through the use of innovative technologies, and the ability to effectively use biomedical instrumentation and the processing and analyzing methods for studying the complex systems and processes of the human body. In order to deepen the acquisition of some omics-related disciplines, the objective of the present work is the development of a laboratory experiment in the field of molecular imaging, whose purpose is the metabolomic study (acquisition and data processing) of biological samples by Nuclear Magnetic Resonance (NMR) spectroscopy. The proposed laboratory experiment is carried out in groups of 15 students and it is structured in two different sessions. The first session is held at the facilities of the Molecular and Metabolomic Imaging Laboratory of the Unitat Central d¿Investigació de Medicina of University of Valencia, and during the session each student performs the sample collection and treatment of their own biological sample (urine), whose metabolomic study is desired, following the appropriate laboratory protocol. After preparation, the acquisition of the NMR spectra corresponding to each of the samples is carried out. All students introduce their samples into the NMR spectrometer and observe the different phases of the acquisition process (frequency tuning and matching, sequence specification, pulse calculation, etc.) to obtain the NMR spectrum. In the second session, which takes place in a computer classroom of the Universitat Politècnica de València, the purpose is the use of different methods of analysis and observation to identify and quantify the metabolites from the NMR spectra acquired during the first session. Firstly, signal processing techniques are used to process the NMR spectra (phase correction, baseline correction and spectrum reference), and then a total of 10 selected metabolites contained in the urine are quantified, using specific software for its application to the field of metabolomics. Finally, the students create a table that collects the data of the subjects (themselves, anonimized) and metabolite concentration, and perform the multivariate statistical analysis: principal component analysis (PCA) and partial least squares discriminant analysis (PLSDA). Through the performed analysis, each student establishes one metabolic profile and obtains conclusions from it, such as the determination of a characteristic metabolomic fingerprint associated with variables such as the sex of the person providing the sample, or the determination of a smoker profile, among others. Through the completion of the proposed laboratory experiment, students will acquire general and specific outcomes in the Biomedical Engineering field: 1) implementation and management of adequate experiments (preparation of biological samples), 2) use of innovative technologies (process of acquisition of NMR spectra), 3) application of scientific methods to perform the analysis of data (signal processing), 4) selection and performance of observation methods (statistical analysis) to assess the results and build conclusions, 5) to carry out a simple metabolic study that allows to verify the metabolic impact of some variable as smoking, and 6) to propose new working hypotheses based on the results obtained and expected results., This work was supported by Conselleria d¿Educació of Generalitat Valenciana [grants number GV/2018/111 and APOSTD/2018/181] and Instituto de Salud Carlos III-FEDER (Fondo Europeo de Desarrollo Regional) [grant number CB16/11/00486].
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Contribución de congreso (ConferenceOutput)

MLLP-VRAIN UPV systems for the IWSLT 2022 Simultaneous Speech Translation and Speech-to-Speech Translation tasks

  • Iranzo-Sánchez, Javier
  • Pérez-González de Martos, Alejandro
  • Baquero-Arnal, Pau
  • Jorge-Cano, Javier|||0000-0002-9279-6768
  • Giménez Pastor, Adrián
  • Garcés Díaz-Munío, Gonçal|||0000-0002-2594-5858
  • Silvestre Cerdà, Joan Albert|||0000-0003-2291-8296
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Sanchis Navarro, José Alberto|||0000-0002-2943-0990
  • Juan, Alfons|||0000-0002-9984-4072
[EN] This work describes the participation of the MLLP-VRAIN research group in the two shared tasks of the IWSLT 2022 conference: Simultaneous Speech Translation and Speech-to-Speech Translation. We present our streaming-ready ASR, MT and TTS systems for Speech Translation and Synthesis from English into German. Our submission combines these systems by means of a cascade approach paying special attention to data preparation and decoding for streaming inference., The research leading to these results has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreements no. 761758 (X5Gon) and 952215 (TAILOR), and Erasmus+ Education programme under grant agreement no. 20- 226-093604-SCH (EXPERT); the Government of Spain¿s grant RTI2018-094879-B-I00 (Multisub) funded by MCIN/AEI/10.13039/501100011033 & ERDF A way of making Europe, and FPU scholarships FPU18/04135; and the Generalitat Valenciana¿s research project Classroom Activity Recognition (ref. PROMETEO/2019/111).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Contribución de congreso (ConferenceOutput)

Europarl-ASR: A Large Corpus of Parliamentary Debates for Streaming ASR Benchmarking and Speech Data Filtering/Verbatimization

  • Garcés Díaz-Munío, Gonçal|||0000-0002-2594-5858
  • Silvestre Cerdà, Joan Albert|||0000-0003-2291-8296
  • Jorge-Cano, Javier|||0000-0002-9279-6768
  • Giménez Pastor, Adrián
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Sanchis Navarro, José Alberto|||0000-0002-2943-0990
  • Juan, Alfons|||0000-0002-9984-4072
  • Iranzo-Sánchez, Javier
  • Baquero-Arnal, Pau
  • Roselló, Nahuel
  • Pérez-González de Martos, Alejandro Manuel
[EN] We introduce Europarl-ASR, a large speech and text corpus of parliamentary debates including 1300 hours of transcribed speeches and 70 million tokens of text in English extracted from European Parliament sessions. The training set is labelled with the Parliament¿s non-fully-verbatim official transcripts, time-aligned. As verbatimness is critical for acoustic model training, we also provide automatically noise-filtered and automatically verbatimized transcripts of all speeches based on speech data filtering and verbatimization techniques. Additionally, 18 hours of transcribed speeches were manually verbatimized to build reliable speaker-dependent and speaker-independent development/test sets for streaming ASR benchmarking. The availability of manual non-verbatim and verbatim transcripts for dev/test speeches makes this corpus useful for the assessment of automatic filtering and verbatimization techniques. This paper describes the corpus and its creation, and provides off-line and streaming ASR baselines for both the speaker-dependent and speaker-independent tasks using the three training transcription sets. The corpus is publicly released under an open licence., [CA] "Europarl-ASR: Un extens corpus parlamentari de referència per a reconeixement de la parla i filtratge/literalització de transcripcions": Presentem Europarl-ASR, un extens corpus de veu i text de debats parlamentaris amb 1300 hores d'intervencions transcrites i 70 milions de paraules de text en anglés extrets de sessions del Parlament Europeu. Les transcripcions oficials del Parlament Europeu, no literals, s'han sincronitzat per a tot el conjunt d'entrenament. Com que l'entrenament de models acústics requereix transcripcions com més literals millor, també s'han inclòs transcripcions filtrades i transcripcions literalitzades de totes les intervencions, basades en tècniques de filtratge i literalització automàtics. A més, s'han inclòs 18 hores de transcripcions literals revisades manualment per definir dos conjunts de validació i avaluació de referència per a reconeixement automàtic de la parla en temps real, amb oradors coneguts i amb oradors desconeguts. Pel fet de disposar de transcripcions literals i no literals, aquest corpus és també ideal per a l'anàlisi de tècniques de filtratge i de literalització. En aquest article, es descriu la creació del corpus i es proporcionen mesures de referència de reconeixement automàtic de la parla en temps real i en diferit, amb oradors coneguts i amb oradors desconeguts, usant els tres conjunts de transcripcions d'entrenament. El corpus es fa públic amb una llicència oberta., This work has received funding from the EU's H2020 research and innovation programme under grant agreements 761758 (X5gon) and 952215 (TAILOR); the Government of Spain's research project Multisub (RTI2018-094879-B-I00, MCIU/AEI/FEDER,EU) and FPU scholarships FPU14/03981 and FPU18/04135; the Generalitat Valenciana's research project Classroom Activity Recognition (PROMETEO/2019/111) and predoctoral research scholarship ACIF/2017/055; and the Universitat Politecnica de València's ` PAID-01-17 R&D support programme.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Contribución de congreso (ConferenceOutput)

Influence of pollution build-up and pavement cross section on permeable pavements under extreme rainfall events

  • Fernández-Gonzalvo, Miriam Inmaculada
  • Hernández Crespo, Carmen|||0000-0002-6727-0481
  • Martín Monerris, Miguel|||0000-0001-7464-9505
  • Andrés-Doménech, Ignacio|||0000-0003-4237-4863
[EN] Permeable pavement systems (PPS) are part of the stormwater management technologies known as Sustainable Urban Drainage Systems (SUDS). The main objective of this is to analyse its buffering capacity (quantity and quality of filtered water) under extreme rainfall events. To this end, two different pavement cross-sections, complete configuration (C1) and simple configuration (C2), have been tested at laboratory scale for the 25-year return period 10-min intensity rainfall in Valencia (Spain). Four different pollutant build-up levels have been studied (at one, three, six and 12 months). For all the simulated rainfall events, the quantity (hydrographs and drained volume) and quality (DQO, TSS, TN and TP concentrations) of filtered water were characterized. On the one hand, configuration C1 reduced more than C2 the peak flow of filtered water for the same dirt and dust build-up. Moreover, peaks are wider in time, so the filtered water is better distributed. Regarding the influence of pollution build-up level, it can be observed that higher dust and dirt build-up gave rise to lower flow rates. In general, the larger the quantity of pollutant build-up accumulated on the pavement surface, the larger the volume of water was retained because it absorbed part of the rain, thus smoothing the peak flow of filtered water. On the other hand, the pollutant mass leached from the pavement C1 is lower than C2. When the amount of dust and dirt deposited was lower, the rainfall was able to leach more pollutant with respect to the total mass deposited. The results show the importance of an adequate cleaning program with a frequency no lower than once a month and a better response, in terms of water pollution retention, of the complete configuration (C1) during times of heavy rainfall., This research has been developed within the SUPRIS-SUPel project framework (Ref. BIA2015-6540-C2-2-R MINECO/FEDER, EU). This project had been financed by the Ministry of Economy and Competitiveness of Spain through the General State Budget and the European Regional Development Fund (ERDF). Currently, the second phase of the project, ENGODRAIN ¿ RTI2018-094217-B-C31, financed by the State Agency for Research from Spain through the General State Budget and the European Regional Development Fund (ERDF), is continuing. The authors would also like to thank the companies Quadro Concrete Prefabs S.L., TenCate Geosynthetics Iberia S.L., Atlantis SUDS S.L. and SECOPSA for supplying materials for the tests. Finally, the hiring of Miriam Inmaculada Fernández Gonzalvo has been possible to the program of subsides for hiring of pre-doctoral research personnel (ACIF/2018/111) of the Generalitat Valenciana.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Contribución de congreso (ConferenceOutput)

Stream-level Latency Evaluation for Simultaneous Machine Translation

  • Iranzo-Sánchez, Javier
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Juan, Alfons|||0000-0002-9984-4072
[EN] Simultaneous machine translation has recently gained traction thanks to significant quality improvements and the advent of streaming applications. Simultaneous translation systems need to find a trade-off between translation quality and response time, and with this purpose multiple latency measures have been proposed. However, latency evaluations for simultaneous translation are estimated at the sentence level, not taking into account the sequential nature of a streaming scenario. Indeed, these sentence-level latency measures are not well suited for continuous stream translation, resulting in figures that are not coherent with the simultaneous translation policy of the system being assessed. This work proposes a stream-level adaptation of the current latency measures based on a re-segmentation approach applied to the output translation, that is successfully evaluated on streaming conditions for a reference IWSLT task., The research leading to these results has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement no. 761758 (X5Gon) and 952215 (TAILOR) and Erasmus+ Education program under grant agreement no. 20-226-093604-SCH; the Government of Spain's research project Multisub, ref. RTI2018-094879-B-I00 (MCIU/AEI/FEDER,EU) and FPU scholarships FPU18/04135; and the Generalitat Valenciana's research project Classroom Activity Recognition, ref. PROMETEO/2019/111.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Contribución de congreso (ConferenceOutput)

Direct Segmentation Models for Streaming Speech Translation

  • Iranzo-Sánchez, Javier
  • Baquero-Arnal, Pau
  • Giménez Pastor, Adrián
  • Silvestre Cerdà, Joan Albert|||0000-0003-2291-8296
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Juan, Alfons|||0000-0002-9984-4072
[EN] The cascade approach to Speech Translation (ST) is based on a pipeline that concatenates an Automatic Speech Recognition (ASR) system followed by a Machine Translation (MT) system. These systems are usually connected by a segmenter that splits the ASR output into, hopefully, semantically self-contained chunks to be fed into the MT system. This is specially challenging in the case of streaming ST, where latency requirements must also be taken into account. This work proposes novel segmentation models for streaming ST that incorporate not only textual, but also acoustic information to decide when the ASR output is split into a chunk. An extensive and thorough experimental setup is carried out on the Europarl-ST dataset to prove the contribution of acoustic information to the performance of the segmentation model in terms of BLEU score in a streaming ST scenario. Finally, comparative results with previous work also show the superiority of the segmentation models proposed in this work., The research leading to these results has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement no. 761758 (X5Gon); the Government of Spain's research project Multisub, ref. RTI2018- 094879-B-I00 (MCIU/AEI/FEDER,EU), the Generalitat Valenciana's research project Classroom Activity Recognition, ref. PROMETEO/2019/111., FPU scholarship FPU18/04135; and the Generalitat Valencianas predoctoral research scholarship ACIF/2017/055. The authors wish to thank the anonymous reviewers for their criticisms and suggestions.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Contribución de congreso (ConferenceOutput)

MLLP-VRAIN Spanish ASR Systems for the Albayzin-RTVE 2020 Speech-To-Text Challenge

  • Jorge-Cano, Javier|||0000-0002-9279-6768
  • Giménez Pastor, Adrián
  • Garcés Díaz-Munío, Gonçal|||0000-0002-2594-5858
  • Silvestre Cerdà, Joan Albert|||0000-0003-2291-8296
  • Civera Saiz, Jorge|||0000-0002-0963-0143
  • Sanchis Navarro, José Alberto|||0000-0002-2943-0990
  • Juan, Alfons|||0000-0002-9984-4072
  • Baquero-Arnal, Pau
  • Iranzo-Sánchez, Javier
  • Pérez-González de Martos, Alejandro Manuel
[EN] This paper describes the automatic speech recognition (ASR) systems built by the MLLP-VRAIN research group of Universitat Politecnica de València for the Albayzin-RTVE 2020 Speech-to-Text Challenge. The primary system (p-streaming_1500ms_nlt) was a hybrid BLSTM-HMM ASR system using streaming one-pass decoding with a context window of 1.5 seconds and a linear combination of an n-gram, a LSTM, and a Transformer language model (LM). The acoustic model was trained on nearly 4,000 hours of speech data from different sources, using the MLLP's transLectures-UPV toolkit (TLK) and TensorFlow; whilst LMs were trained using SRILM (n-gram), CUED-RNNLM (LSTM) and Fairseq (Transformer), with up to 102G tokens. This system achieved 11.6% and 16.0% WER on the test-2018 and test-2020 sets, respectively. As it is streaming-enabled, it could be put into production environments for automatic captioning of live media streams, with a theoretical delay of 1.5 seconds. Along with the primary system, we also submitted three contrastive systems. From these, we highlight the system c2-streaming_600ms_t that, following the same configuration of the primary one, but using a smaller context window of 0.6 seconds and a Transformer LM, scored 12.3% and 16.9% WER points respectively on the same test sets, with a measured empirical latency of 0.81+-0.09 seconds (mean+-stdev). This is, we obtained state-of-the-art latencies for high-quality automatic live captioning with a small WER degradation of 6% relative., [CA] En aquest article, es descriuen els sistemes de reconeixement automàtic de la parla (RAP) creats pel grup d'investigació MLLP-VRAIN de la Universitat Politecnica de València per a la competició Albayzin-RTVE 2020 Speech-to-Text Challenge. El sistema primari (p-streaming_1500ms_nlt) és un sistema de RAP híbrid BLSTM-HMM amb descodificació en temps real en una passada amb una finestra de context d'1,5 segons i una combinació lineal de models de llenguatge (ML) d'n-grames, LSTM i Transformer. El model acústic s'ha entrenat amb vora 4000 hores de parla transcrita de diferents fonts, usant el transLectures-UPV toolkit (TLK) del grup MLLP i TensorFlow; mentre que els ML s'han entrenat amb SRILM (n-grames), CUED-RNNLM (LSTM) i Fairseq (Transformer), amb 102G paraules (tokens). Aquest sistema ha obtingut 11,6 % i 16,0 % de WER en els conjunts test-2018 i test-2020, respectivament. És un sistema amb capacitat de temps real, que pot desplegar-se en producció per a subtitulació automàtica de fluxos audiovisuals en directe, amb un retard teòric d'1,5 segons. A banda del sistema primari, s'han presentat tres sistemes contrastius. D'aquests, destaquem el sistema c2-streaming_600ms_t que, amb la mateixa configuració que el sistema primari, però amb una finestra de context més reduïda de 0,6 segons i un ML Transformer, ha obtingut 12,3 % i 16,9 % de WER, respectivament, sobre els mateixos conjunts, amb una latència empírica mesurada de 0,81+-0,09 segons (mitjana+-desv). És a dir, s'han obtingut latències punteres per a subtitulació automàtica en directe d'alta qualitat amb una degradació del WER petita, del 6 % relatiu., The research leading to these results has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement no. 761758 (X5Gon); the Government of Spain¿s research project Multisub (ref. RTI2018-094879-B-I00, MCIU/AEI/FEDER,EU) and FPU scholarships FPU14/03981 and FPU18/04135; and the Generalitat Valenciana¿s research project Classroom Activity Recognition (ref. PROMETEO/2019/111) and predoctoral research scholarship ACIF/2017/055
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Automatic piano fingering from partially annotated scores using autoregressive neural networks

  • Ramoneda, Pedro
  • Jeong, Dasaem
  • Nakamura, Eita
  • Serra, Xavier
  • Miron, Marius
Comunicació presentada a: 30th ACM International Conference on Multimedia (MM'22), celebrat del 10 al 14 d'octubre de 2022 a Lisboa, Portugal, Piano fingering is a creative and highly individualised task acquired by musicians progressively in their first music education years. Pianists must learn to choose the order of fingers to play the piano keys because scores do not have engraved finger and hand movements as other technique elements. Numerous research efforts have been conducted for automatic piano fingering based on a previous dataset composed of 150 score excerpts fully annotated by multiple expert annotators. However, most piano sheets include partial annotations for problematic finger and hand movements. We introduce a novel dataset for the task, the ThumbSet dataset, containing 2523 pieces with partial and noisy annotations of piano fingering crowdsourced from non-expert annotators. As part of our methodology, we propose two autoregressive neural networks with beam search decoding for modelling automatic piano fingering as a sequence-to-sequence learning problem, considering the correlation between output finger labels. We design the first model with the exact pitch representation of previous proposals. The second model uses graph neural networks to more effectively represent polyphony, whose treatment has been a common issue across previous studies. Finally, we finetune the models on the existing expert annotations dataset. The evaluation shows that (1) we are able to achieve high performance when training on the ThumbSet dataset and that (2) the proposed models outperform the state-of-the-art hidden Markov models and recurrent neural network baselines. Code, dataset, models, and results are made available to enhance the task reproducibility, including a new framework for evaluation, This work is supported in part by the project Musical AI - PID2019- 111403GB-I00/AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovacion y Universidades (MCIU) and the Agencia Estatal de Investigacion (AEI), Sogang University Research Grant of 202110035.01 and JSPS KAKENHI Nos. 21K12187 and 22H03661.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Analysis of musical dynamics in vocal performances using loudness measures

  • Narang, Jyoti
  • Miron, Marius
  • Srinivasamurthy, Ajay
  • Serra, Xavier
Comunicació presentada a: DAFx20in22, celebrat del 6 al 10 de setembre de 2022, a Viena, Austria., In addition to tone, pitch and rhythm, dynamics is one of the expressive dimensions of the performance of a music piece that has received limited attention. While the usage of dynamics may vary from artist to artist, and also from performance to performance, a systematic methodology to automatically identify the dynamics of a performance in terms of musically meaningful terms like forte, piano may offer valuable feedback in the context of music education and in particular in singing. To this end, we have manually annotated the dynamic markings of commercial recordings of popular rock and pop songs from the Smule Vocal Balanced (SVB) dataset which will be used as reference data. Then as a first step for our research goal, we propose a method to derive and compare singing voice loudness curves in polyphonic mixtures. Towards measuring the similarity and variation of dynamics, we compare the dynamics curves of the SVB renditions with the one derived from the original songs. We perform the same comparison using professionally produced renditions from a karaoke website. We relate high values of Spearman correlation coefficient found in some select student renditions and the professional renditions with accurate dynamics., Part of this research is funded by Project Musical AI - PID2019-111403GBI00/ AEI/10.13039/501100011033 funded by the Spanish Ministerio de Ciencia, Innovación y Universidades (MCIU) and the Agencia Estatal de Investigación (AEI).