Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 24
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Shall we learn together in loud spaces? Towards understanding the effects of sound in collaborative learning environments

  • Vujovic, Milica
  • Hernández Leo, Davinia
Comunicació presentada a: the 13th International Conference on Computer Supported Collaborative Learning (CSCL), celebrada del 17 al 21 de juny de 2019 a Lió, França., In this paper we question the role of environmental sound on the process of CL. The first pilot study is presented where we investigated effects of environmental sound on EDA and voice VA of the participants. The created visualization presents the dependence between mentioned parameters and serves as an awareness tool for participants in CL. Preliminary results are provocative; there seems to be mentioned dependences and participants accept the proposed visualization as a useful tool to support self-regulation during CL., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. Milica Vujovic has received financial support through the “la Caixa” INPhINIT Fellowship Grant for Doctoral studies at Spanish Research Centres of Excellence, “la Caixa” Banking Foundation, Barcelona, Spain. This work has been partially supported by the National Research Agency of the Spanish Ministry of Science, Innovations and Universities MDM-2015-0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Updateable inner product argument with logarithmic verifier and applications

  • Daza, Vanesa
  • Ràfols, Carla
  • Zacharakis, Alexandros
Comunicació presentada a: 23rd IACR International Conference on Practice and Theory of Public-Key Cryptography celebrat del 4 al 7 de maig de 2020 a Edimburg, Escòcia., We propose an improvement for the inner product argument of Bootle et al. (EUROCRYPT’16). The new argument replaces the unstructured common reference string (the commitment key) by a structured one. We give two instantiations of this argument, for two different distributions of the CRS. In the designated verifier setting, this structure can be used to reduce verification from linear to logarithmic in the circuit size. The argument can be compiled to the publicly verifiable setting in asymmetric bilinear groups. The new common reference string can easily be updateable. The argument can be directly used to improve verification of Bulletproofs range proofs (IEEE SP’18). On the other hand, to use the improved argument to prove circuit satisfiability with logarithmic verification, we adapt recent techniques from Sonic (ACM CCS’19) to work with the new common reference string. The resulting argument is secure under standard assumptions (in the Random Oracle Model), in contrast with Sonic and recent works that improve its efficiency (Plonk, Marlin, AuroraLight), which, apart from the Random Oracle Model, need either the Algebraic Group Model or Knowledge Type assumptions., The project that gave rise to these results received the support of a fellowship from “la Caixa” Foundation (ID 100010434). The fellowship code is LCF/BQ/DI18/11660053. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. First author was supported by Project RTI2018-102112-B-I00 (AEI/FEDER,UE) and this paper is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 856879.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

VocaLiST: an audio-visual synchronisation model for lips and voices

  • Kadandale, Venkatesh S.
  • Montesinos García, Juan Felipe
  • Haro Ortega, Gloria
Comunicació presentada a Interspeech 2022, celebrat del 18 al 22 de setembre de 2022 a Inchon, Corea del Sud., In this paper, we address the problem of lip-voice synchronisation in videos containing human face and voice. Our approach is based on determining if the lips motion and the voice in a video are synchronised or not, depending on their audiovisual correspondence score. We propose an audio-visual crossmodal transformer-based model that outperforms several baseline models in the audio-visual synchronisation task on the standard lip-reading speech benchmark dataset LRS2. While the existing methods focus mainly on lip synchronisation in speech videos, we also consider the special case of the singing voice. The singing voice is a more challenging use case for synchronisation due to sustained vowel sounds. We also investigate the relevance of lip synchronisation models trained on speech datasets in the context of singing voice. Finally, we use the frozen visual features learned by our lip synchronisation model in the singing voice separation task to outperform a baseline audio-visual model which was trained end-to-end. The demos, source code, and the pre-trained models are available on https://ipcv.github.io/VocaLiST/, We acknowledge support by MICINN/FEDER UE project PID2021-127643NB-I00; H2020-MSCA-RISE-2017 project 777826 NoMADS. V. S. K. has received support through “la Caixa” Foundation (ID 100010434), fellowship code: LCF/BQ/DI18/11660064 and the Marie SkłodowskaCurie grant agreement No. 713673. J. F. M. acknowledges support by FPI scholarship PRE2018-083920.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Fractional homomorphism, Weisfeiler-Leman invariance, and the Sherali-Adams hierarchy for the constraint satisfaction problem

  • Butti, Silvia
  • Dalmau, Víctor
Comunicació presentada a: MFCS 2021 celebrat del 23 a 27 d'agost de 2021 a Tallinn, Estònia., Given a pair of graphs 𝐀 and 𝐁, the problems of deciding whether there exists either a homomorphism or an isomorphism from 𝐀 to 𝐁 have received a lot of attention. While graph homomorphism is known to be NP-complete, the complexity of the graph isomorphism problem is not fully understood. A well-known combinatorial heuristic for graph isomorphism is the Weisfeiler-Leman test together with its higher order variants. On the other hand, both problems can be reformulated as integer programs and various LP methods can be applied to obtain high-quality relaxations that can still be solved efficiently. We study so-called fractional relaxations of these programs in the more general context where 𝐀 and 𝐁 are not graphs but arbitrary relational structures. We give a combinatorial characterization of the Sherali-Adams hierarchy applied to the homomorphism problem in terms of fractional isomorphism. Collaterally, we also extend a number of known results from graph theory to give a characterization of the notion of fractional isomorphism for relational structures in terms of the Weisfeiler-Leman test, equitable partitions, and counting homomorphisms from trees. As a result, we obtain a description of the families of CSPs that are closed under Weisfeiler-Leman invariance in terms of their polymorphisms as well as decidability by the first level of the Sherali-Adams hierarchy., Silvia Butti: The project that gave rise to these results received the support of a fellowship from “la Caixa” Foundation (ID 100010434). The fellowship code is LCF/BQ/DI18/11660056. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. Víctor Dalmau: Víctor Dalmau was supported by MICCIN grant PID2019-109137GB-C22.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Medical-based deep curriculum learning for improved fracture classification

  • Jiménez Sánchez, Amelia
  • Mateus, Diana
  • Kirchhoff, Sonja
  • Kirchhoff, Chlodwig
  • Biberthaler, Peter
  • Navab, Nassir
  • González Ballester, Miguel Ángel, 1973-
  • Piella Fenoy, Gemma
Comunicació presentada al MICCAI 2019: Medical Image Computing and Computer Assisted Intervention, celebrat del 13 al 17 d'octubre de 2019 a Shenzhen, Xina., Current deep-learning based methods do not easily integrate to clinical protocols, neither take full advantage of medical knowledge. In this work, we propose and compare several strategies relying on curriculum learning, to support the classification of proximal femur fracture from X-ray images, a challenging problem as reflected by existing intra- and inter-expert disagreement. Our strategies are derived from knowledge such as medical decision trees and inconsistencies in the annotations of multiple experts, which allows us to assign a degree of difficulty to each training sample. We demonstrate that if we start learning “easy” examples and move towards “hard”, the model can reach a better performance, even with fewer data. The evaluation is performed on the classification of a clinical dataset of about 1000 X-ray images. Our results show that, compared to class-uniform and random strategies, the proposed medical knowledge-based curriculum, performs up to 15% better in terms of accuracy, achieving the performance of experienced trauma surgeons., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673 and by the Spanish Ministry of Economy [MDM-2015-0502]. A. Jiménez-Sánchez has received financial support through the “la Caixa” Foundation (ID Q5850017D), fellowship code: LCF/BQ/IN17/11620013. D. Mateus has received funding from Nantes Métropole and the European Regional Development, Pays de la Loire, under the Connect Talent scheme. Authors thank Nvidia for the donation of a GPU.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

The complexity of the distributed constraint satisfaction problem

  • Butti, Silvia
  • Dalmau, Víctor
Comunicació presentada a: STACS 2021 celebrat del 16 a 19 de març de 2021 a Saarbrücken, Alemanya., We study the complexity of the Distributed Constraint Satisfaction Problem (DCSP) on a synchronous, anonymous network from a theoretical standpoint. In this setting, variables and constraints are controlled by agents which communicate with each other by sending messages through fixed communication channels. Our results endorse the well-known fact from classical CSPs that the complexity of fixed-template computational problems depends on the template’s invariance under certain operations. Specifically, we show that DCSP(Γ) is polynomial-time tractable if and only if Γ is invariant under symmetric polymorphisms of all arities. Otherwise, there are no algorithms that solve DCSP(Γ) in finite time. We also show that the same condition holds for the search variant of DCSP. Collaterally, our results unveil a feature of the processes’ neighbourhood in a distributed network, its iterated degree, which plays a major role in the analysis. We explore this notion establishing a tight connection with the basic linear programming relaxation of a CSP., The project that gave rise to these results received the support of a fellowship from “la Caixa” Foundation (ID 100010434). The fellowship code is LCF/BQ/DI18/11660056. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sk lodowska-Curie grant agreement No. 713673. Victor Dalmau was supported by MICCIN grants TIN2016-76573-C2-1P and PID2019-109137GB-C22.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Photophysiology of plants with divergent responses to vegetation proximity

  • Morelli, Luca
  • Paulišić, Sandi
  • Martínez-García, Jaime F.
  • Rodriguez-Concepcion, Manuel
Póster presentado a la XXIII Reunión Bianual de la Sociedad Española de Fisiología Vegetal y al XVI Congreso Hispano-Luso de Fisiología Vegetal, celebrados en Pamplona (España) del 26 al 28 de junio de 2019., This work was funded by grants BIO2017-90877-REDT, BIO2017-85316-R and BIO2017-84041-P from the Spanish Ministry of Science, Innovation, and Universities (MICINN) and grants 2017SGR-1211 and 2017SGR-710 from Generalitat de Catalunya to JFMG and MRC. We thank the financial support of the MINECO Severo Ochoa Programme for Centres of Excellence in R&D 2016-2019 (SEV‐2015‐0533) and the Generalitat de Catalunya CERCA Programme to CRAG. LM was supported by “la Caixa” Foundation (ID 100010434) with the PhD INPhINIT fellowship LCF/BQ/IN18/11660004. This project has received funding from the European Union’s Horizon 2020 research and in-novation programme under the Marie Skłodowska-Curie grant agreement No. 713673.
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Combined MRI-Microbiota analysis in Alcohol Use Disorders

  • Selim, Mohamed
  • De Santis, Silvia
  • Sanz Herranz, Yolanda
  • Benítez-Páez, Alfonso
  • Ciccocioppo, Roberto
  • Canals, Santiago
Trabajo presentado al 12th Federation of European Neuroscience Societies (FENS): Virtual Forum of NeuroScience, celebrado del 11 al 15 de julio de 2020., M.S.issupported by La Caixa Foundation INPhINT Fellowship (LCF/BQ/IN18/11660067), whichr eceived funding from EU-H2020 (MSCA-COFUNDNo.713673). Project received funding from Generalitat Valencia (PROMETEO/2019/015), Ministerio de Sanidad , Servicios Sociales e Igualdad (#2017I065) and Severo Ochoa Program for Centers of Excellence in R&D (SEV-2017-0723)., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Dynamic time warping analysis of the evolution of SMOS surfaceand in-situ soil moisture time series

  • Herbert, Christoph
  • Pablos, Miriam
  • Vall-llossera, Mercè
  • Camps, Adriano
European Geosciences Union (EGU) General Assembly 2020, 4-8 May 2020.-- 1 page, figures, A comprehensive understanding of temporal variability of root-zone and surface soil moisture (SM) and the relationship with the underlying soil characteristics is of great importance in hydrological and agricultural applications. For the last ten years, global and frequent satellite SM observations have been available to investigate SM dynamics. However, validating remote sensing retrievals against in-situ observations based on the comparison of collocated SM time series is complicated. While satellite retrievals are approximated from inversion models over an area, in-situ measurements are determined at point-scale. This usually produces different SM dynamic ranges and biases in the corresponding time series. Moreover, the influence of soil properties and meteorological conditions can cause SM time series obtained from indirect remote sensing techniques and direct in-situ observations to be non-linearly related. Dynamic Time Warping (DTW) is a dynamic programming technique, capable of coping with temporal distortions by aiming for finding the optimal match between time series. In this study, DTW was used to provide a time lag evolution as a continuous dissimilarity measure comprising the main temporal variability features of two time series. The DTW technique was applied to SM time series from the Soil Moisture and Ocean Salinity mission (SMOS) L4 product developed at Barcelona Expert Center (BEC) with in-situ measurements at top- and subsoil-representative depth levels, located in the Soil Moisture Measurements Station Network of the University of Salamanca (REMEDHUS) in Western Spain. DTW parameters were customized to the particular input time series to obtain a robust and meaningful time lag. Seasonal differences in SM dynamics were analyzed in a clustering approach by investigating the link between SM time series and SM-regime-related parameters including precipitation and categorical features such as soil type and land use. Since the technique resolves the non-linear behaviour of time series, it has the potential to generally assess major differences in SM acquisition techniques. It could also be useful to investigate spatial SM variability in heterogeneous regions and to make informed choices in future sensor deployment in SM networks, The project that gave rise to these results received the support of a fellowship from “la Caixa” Foundation (ID 100010434). The fellowship code is LCF/BQ/DI18/11660050. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 713673. This study was also funded through the award "Unidad de Excelencia María de Maeztu” MDM-2016-060, Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Contribución de congreso (ConferenceOutput)

Axonal pathology in alcohol use disorders

  • Selim, Mohamed
  • De Santis, Silvia
  • Ciccocioppo, Roberto
  • Canals, Santiago
Poster presentado en el congreso del FENS (Federation of European Neuroscience Societies), celebrado en Paris(France) del 9 al 13 de julio de 2022, M.K.S. is supported by La Caixa Foundation INPhINT Fellowship (LCF/BQ/IN18/11660067), which received funding from EU-H2020 (MSCA-COFUND No. 713673 )., Peer reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Optimization of the progress variable definition using a genetic algorithm for the combustion of complex fuels

  • Both, Ambrus|||0000-0002-5267-204X
  • Mira Martínez, Daniel
  • Lehmkuhl Barba, Oriol|||0000-0002-2670-1871
In this work counterflow diffusion flamelets of n-heptane and air are used at stable and unsteady extinguishing conditions for building a thermo-chemical database for Computational Fluid Dynamics (CFD) calculations. The injectivity of the progress variable definition is achieved through an optimization process using a genetic algorithm in combination with an adequate objective function., The research activities conducted here have been financed by the CoEC project from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 952181. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. Ambrus Both has received financial support from "la Caixa" Fundation (ID 100010434). The fellowship grant code is LCF/BQ/IN17/11620032. We also acknowledge Red Española de Supercom- putación (RES) for awarding us access to the MareNostrum IV machine based in Barcelona, Spain (Ref. IM-2020-2-0025), Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

An oracle for guiding large-scale model/hybrid parallel training of convolutional neural networks

  • Njoroge Kahira, Albert|||0000-0002-1138-0577
  • Nguyen, Truong Thao
  • Bautista Gomez, Leonardo|||0000-0002-0814-5779
  • Takano, Ryousei
  • Badia Sala, Rosa Maria|||0000-0003-2941-5499
  • Wahib, Mohamed
Deep Neural Network (DNN) frameworks use distributed training to enable faster time to convergence and alleviate memory capacity limitations when training large models and/or using high dimension inputs. With the steady increase in datasets and model sizes, model/hybrid parallelism is deemed to have an important role in the future of distributed training of DNNs. We analyze the compute, communication, and memory requirements of Convolutional Neural Networks (CNNs) to understand the trade-offs between different parallelism approaches on performance and scalability. We leverage our model-driven analysis to be the basis for an oracle utility which can help in detecting the limitations and bottlenecks of different parallelism approaches at scale. We evaluate the oracle on six parallelization strategies, with four CNN models and multiple datasets (2D and 3D), on up to 1024 GPUs. The results demonstrate that the oracle has an average accuracy of about 86.74% when compared to empirical results, and as high as 97.57% for data parallelism., The project that gave rise to these results received the support of a fellowship from the "la Caixa" Foundation (ID 100010434). The fellowship code is LCF/BQ/DI17/11620059. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. The Eurolab4HPC project has received funding from the European Union Horizon 2020 Framework Programme (H2020-EU.1.2.2. - FET Proactive) under grant agreement number 800962. This work was supported by JST, ACT-X Grant Number JPMJAX190C, Japan; by JST, PRESTO Grant Number JPMJPR20MA, Japan., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Wav2Pix: speech-conditioned face generation using generative adversarial networks

  • Cardoso Duarte, Amanda
  • Roldan, Francisco
  • Tubau, Miquel
  • Escur, Janna
  • Pascual de la Puente, Santiago|||0000-0002-8365-7387
  • Salvador Aguilera, Amaia|||0000-0002-9908-1685
  • Mohedano, Eva
  • McGuinness, Kevin
  • Torres Viñals, Jordi|||0000-0003-1963-7418
  • Giró Nieto, Xavier|||0000-0002-9935-5332
Speech is a rich biometric signal that contains information about the identity, gender and emotional state of the speaker. In this work, we explore its potential to generate face images of a speaker by conditioning a Generative Adversarial Network (GAN) with raw speech input. We propose a deep neural network that is trained from scratch in an end-to-end fashion, generating a face directly from the raw speech waveform without any additional identity information (e.g reference image or one-hot encoding). Our model is trained in a self-supervised approach by exploiting the audio and visual signals naturally aligned in videos. With the purpose of training from video data, we present a novel dataset collected for this work, with high-quality videos of youtubers with notable expressiveness in both the speech and visual signals., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Contribución de congreso (ConferenceOutput)

Probabilistic inference method to discriminate closed water from sea ice using SENTINEL-1 Sar signatures

  • Herbert, Christoph Josef|||0000-0002-2578-8522
  • Camps Carmona, Adriano José|||0000-0002-9514-4992
  • Vall-Llossera Ferran, Mercedes Magdalena|||0000-0003-1357-7098
Consistent sea ice monitoring requires accurate estimates of sea ice concentration. Current retrieval algorithms are based on low-resolution microwave radiometry data with limited penetration depth and are unable to resolve surface characteristics of sea ice in sufficient detail which is necessary to discriminate intact sea ice from closed water. Important information about surface roughness conditions are contained in the distribution of radar backscattering images which can be - in principle - used to detect melt ponds and different sea ice types. In this work, a two-step probabilistic approach based on Expectation-Maximization and Bayesian inference considers the spatial and statistical characteristics of medium-resolution daily-available Sentinel-1 SAR images. The presented method segments sea ice into a number of separable classes and enables to discriminate surface water from the remaining sea ice types., The lead author was supported by “la Caixa” Foundation (ID 100010434) with the fellowship code LCF/BQ/D118/11660050, and received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 713673. The project was also funded through the award “Unidad de Excelencia María de Maeztu” MDM-2016-0600, by the Spanish Ministry of Science and Innovation through the project “L-band” ESP2017-89463-C3-2-R, and the project “Sensing with Pioneering Opportunistic Techniques (SPOT)” RTI2018-099008-B-C21/AEI/10.13039/501100011033., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Fractional homomorphism, Weisfeiler-Leman invariance, and the Sherali-Adams hierarchy for the constraint satisfaction problem

  • Butti, Silvia
  • Dalmau, Víctor
Comunicació presentada a: MFCS 2021 celebrat del 23 a 27 d'agost de 2021 a Tallinn, Estònia., Given a pair of graphs 𝐀 and 𝐁, the problems of deciding whether there exists either a homomorphism or an isomorphism from 𝐀 to 𝐁 have received a lot of attention. While graph homomorphism is known to be NP-complete, the complexity of the graph isomorphism problem is not fully understood. A well-known combinatorial heuristic for graph isomorphism is the Weisfeiler-Leman test together with its higher order variants. On the other hand, both problems can be reformulated as integer programs and various LP methods can be applied to obtain high-quality relaxations that can still be solved efficiently. We study so-called fractional relaxations of these programs in the more general context where 𝐀 and 𝐁 are not graphs but arbitrary relational structures. We give a combinatorial characterization of the Sherali-Adams hierarchy applied to the homomorphism problem in terms of fractional isomorphism. Collaterally, we also extend a number of known results from graph theory to give a characterization of the notion of fractional isomorphism for relational structures in terms of the Weisfeiler-Leman test, equitable partitions, and counting homomorphisms from trees. As a result, we obtain a description of the families of CSPs that are closed under Weisfeiler-Leman invariance in terms of their polymorphisms as well as decidability by the first level of the Sherali-Adams hierarchy., Silvia Butti: The project that gave rise to these results received the support of a fellowship from “la Caixa” Foundation (ID 100010434). The fellowship code is LCF/BQ/DI18/11660056. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. Víctor Dalmau: Víctor Dalmau was supported by MICCIN grant PID2019-109137GB-C22.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Optimization of the progress variable definition using a genetic algorithm for the combustion of complex fuels

  • Both, Ambrus
  • Mira Martínez, Daniel
  • Lehmkuhl Barba, Oriol
In this work counterflow diffusion flamelets of n-heptane and air are used at stable and unsteady extinguishing conditions for building a thermo-chemical database for Computational Fluid Dynamics (CFD) calculations. The injectivity of the progress variable definition is achieved through an optimization process using a genetic algorithm in combination with an adequate objective function., The research activities conducted here have been financed by the CoEC project from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 952181. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. Ambrus Both has received financial support from "la Caixa" Fundation (ID 100010434). The fellowship grant code is LCF/BQ/IN17/11620032. We also acknowledge Red Española de Supercom- putación (RES) for awarding us access to the MareNostrum IV machine based in Barcelona, Spain (Ref. IM-2020-2-0025), Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Wav2Pix: speech-conditioned face generation using generative adversarial networks

  • Cardoso Duarte, Amanda
  • Roldan, Francisco
  • Tubau, Miquel
  • Escur, Janna
  • Pascual de la Puente, Santiago
  • Salvador Aguilera, Amaia
  • Mohedano, Eva
  • McGuinness, Kevin
  • Torres Viñals, Jordi
  • Giró Nieto, Xavier
Speech is a rich biometric signal that contains information about the identity, gender and emotional state of the speaker. In this work, we explore its potential to generate face images of a speaker by conditioning a Generative Adversarial Network (GAN) with raw speech input. We propose a deep neural network that is trained from scratch in an end-to-end fashion, generating a face directly from the raw speech waveform without any additional identity information (e.g reference image or one-hot encoding). Our model is trained in a self-supervised approach by exploiting the audio and visual signals naturally aligned in videos. With the purpose of training from video data, we present a novel dataset collected for this work, with high-quality videos of youtubers with notable expressiveness in both the speech and visual signals., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

VocaLiST: an audio-visual synchronisation model for lips and voices

  • Kadandale, Venkatesh S.
  • Montesinos García, Juan Felipe
  • Haro Ortega, Gloria
Comunicació presentada a Interspeech 2022, celebrat del 18 al 22 de setembre de 2022 a Inchon, Corea del Sud., In this paper, we address the problem of lip-voice synchronisation in videos containing human face and voice. Our approach is based on determining if the lips motion and the voice in a video are synchronised or not, depending on their audiovisual correspondence score. We propose an audio-visual crossmodal transformer-based model that outperforms several baseline models in the audio-visual synchronisation task on the standard lip-reading speech benchmark dataset LRS2. While the existing methods focus mainly on lip synchronisation in speech videos, we also consider the special case of the singing voice. The singing voice is a more challenging use case for synchronisation due to sustained vowel sounds. We also investigate the relevance of lip synchronisation models trained on speech datasets in the context of singing voice. Finally, we use the frozen visual features learned by our lip synchronisation model in the singing voice separation task to outperform a baseline audio-visual model which was trained end-to-end. The demos, source code, and the pre-trained models are available on https://ipcv.github.io/VocaLiST/, We acknowledge support by MICINN/FEDER UE project PID2021-127643NB-I00; H2020-MSCA-RISE-2017 project 777826 NoMADS. V. S. K. has received support through “la Caixa” Foundation (ID 100010434), fellowship code: LCF/BQ/DI18/11660064 and the Marie SkłodowskaCurie grant agreement No. 713673. J. F. M. acknowledges support by FPI scholarship PRE2018-083920.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Medical-based deep curriculum learning for improved fracture classification

  • Jiménez Sánchez, Amelia
  • Mateus, Diana
  • Kirchhoff, Sonja
  • Kirchhoff, Chlodwig
  • Biberthaler, Peter
  • Navab, Nassir
  • González Ballester, Miguel Ángel, 1973-
  • Piella Fenoy, Gemma
Comunicació presentada al MICCAI 2019: Medical Image Computing and Computer Assisted Intervention, celebrat del 13 al 17 d'octubre de 2019 a Shenzhen, Xina., Current deep-learning based methods do not easily integrate to clinical protocols, neither take full advantage of medical knowledge. In this work, we propose and compare several strategies relying on curriculum learning, to support the classification of proximal femur fracture from X-ray images, a challenging problem as reflected by existing intra- and inter-expert disagreement. Our strategies are derived from knowledge such as medical decision trees and inconsistencies in the annotations of multiple experts, which allows us to assign a degree of difficulty to each training sample. We demonstrate that if we start learning “easy” examples and move towards “hard”, the model can reach a better performance, even with fewer data. The evaluation is performed on the classification of a clinical dataset of about 1000 X-ray images. Our results show that, compared to class-uniform and random strategies, the proposed medical knowledge-based curriculum, performs up to 15% better in terms of accuracy, achieving the performance of experienced trauma surgeons., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673 and by the Spanish Ministry of Economy [MDM-2015-0502]. A. Jiménez-Sánchez has received financial support through the “la Caixa” Foundation (ID Q5850017D), fellowship code: LCF/BQ/IN17/11620013. D. Mateus has received funding from Nantes Métropole and the European Regional Development, Pays de la Loire, under the Connect Talent scheme. Authors thank Nvidia for the donation of a GPU.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Updateable inner product argument with logarithmic verifier and applications

  • Daza, Vanesa
  • Ràfols, Carla
  • Zacharakis, Alexandros
Comunicació presentada a: 23rd IACR International Conference on Practice and Theory of Public-Key Cryptography celebrat del 4 al 7 de maig de 2020 a Edimburg, Escòcia., We propose an improvement for the inner product argument of Bootle et al. (EUROCRYPT’16). The new argument replaces the unstructured common reference string (the commitment key) by a structured one. We give two instantiations of this argument, for two different distributions of the CRS. In the designated verifier setting, this structure can be used to reduce verification from linear to logarithmic in the circuit size. The argument can be compiled to the publicly verifiable setting in asymmetric bilinear groups. The new common reference string can easily be updateable. The argument can be directly used to improve verification of Bulletproofs range proofs (IEEE SP’18). On the other hand, to use the improved argument to prove circuit satisfiability with logarithmic verification, we adapt recent techniques from Sonic (ACM CCS’19) to work with the new common reference string. The resulting argument is secure under standard assumptions (in the Random Oracle Model), in contrast with Sonic and recent works that improve its efficiency (Plonk, Marlin, AuroraLight), which, apart from the Random Oracle Model, need either the Algebraic Group Model or Knowledge Type assumptions., The project that gave rise to these results received the support of a fellowship from “la Caixa” Foundation (ID 100010434). The fellowship code is LCF/BQ/DI18/11660053. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. First author was supported by Project RTI2018-102112-B-I00 (AEI/FEDER,UE) and this paper is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 856879.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Probabilistic inference method to discriminate closed water from sea ice using SENTINEL-1 Sar signatures

  • Herbert, Christoph Josef
  • Camps Carmona, Adriano José
  • Vall-Llossera Ferran, Mercedes Magdalena
Consistent sea ice monitoring requires accurate estimates of sea ice concentration. Current retrieval algorithms are based on low-resolution microwave radiometry data with limited penetration depth and are unable to resolve surface characteristics of sea ice in sufficient detail which is necessary to discriminate intact sea ice from closed water. Important information about surface roughness conditions are contained in the distribution of radar backscattering images which can be - in principle - used to detect melt ponds and different sea ice types. In this work, a two-step probabilistic approach based on Expectation-Maximization and Bayesian inference considers the spatial and statistical characteristics of medium-resolution daily-available Sentinel-1 SAR images. The presented method segments sea ice into a number of separable classes and enables to discriminate surface water from the remaining sea ice types., The lead author was supported by “la Caixa” Foundation (ID 100010434) with the fellowship code LCF/BQ/D118/11660050, and received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 713673. The project was also funded through the award “Unidad de Excelencia María de Maeztu” MDM-2016-0600, by the Spanish Ministry of Science and Innovation through the project “L-band” ESP2017-89463-C3-2-R, and the project “Sensing with Pioneering Opportunistic Techniques (SPOT)” RTI2018-099008-B-C21/AEI/10.13039/501100011033., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Shall we learn together in loud spaces? Towards understanding the effects of sound in collaborative learning environments

  • Vujovic, Milica
  • Hernández Leo, Davinia
Comunicació presentada a: the 13th International Conference on Computer Supported Collaborative Learning (CSCL), celebrada del 17 al 21 de juny de 2019 a Lió, França., In this paper we question the role of environmental sound on the process of CL. The first pilot study is presented where we investigated effects of environmental sound on EDA and voice VA of the participants. The created visualization presents the dependence between mentioned parameters and serves as an awareness tool for participants in CL. Preliminary results are provocative; there seems to be mentioned dependences and participants accept the proposed visualization as a useful tool to support self-regulation during CL., This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. Milica Vujovic has received financial support through the “la Caixa” INPhINIT Fellowship Grant for Doctoral studies at Spanish Research Centres of Excellence, “la Caixa” Banking Foundation, Barcelona, Spain. This work has been partially supported by the National Research Agency of the Spanish Ministry of Science, Innovations and Universities MDM-2015-0502, TIN2014-53199-C3-3-R, TIN2017-85179-C3-3-R.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

The complexity of the distributed constraint satisfaction problem

  • Butti, Silvia
  • Dalmau, Víctor
Comunicació presentada a: STACS 2021 celebrat del 16 a 19 de març de 2021 a Saarbrücken, Alemanya., We study the complexity of the Distributed Constraint Satisfaction Problem (DCSP) on a synchronous, anonymous network from a theoretical standpoint. In this setting, variables and constraints are controlled by agents which communicate with each other by sending messages through fixed communication channels. Our results endorse the well-known fact from classical CSPs that the complexity of fixed-template computational problems depends on the template’s invariance under certain operations. Specifically, we show that DCSP(Γ) is polynomial-time tractable if and only if Γ is invariant under symmetric polymorphisms of all arities. Otherwise, there are no algorithms that solve DCSP(Γ) in finite time. We also show that the same condition holds for the search variant of DCSP. Collaterally, our results unveil a feature of the processes’ neighbourhood in a distributed network, its iterated degree, which plays a major role in the analysis. We explore this notion establishing a tight connection with the basic linear programming relaxation of a CSP., The project that gave rise to these results received the support of a fellowship from “la Caixa” Foundation (ID 100010434). The fellowship code is LCF/BQ/DI18/11660056. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sk lodowska-Curie grant agreement No. 713673. Victor Dalmau was supported by MICCIN grants TIN2016-76573-C2-1P and PID2019-109137GB-C22.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

An oracle for guiding large-scale model/hybrid parallel training of convolutional neural networks

  • Njoroge Kahira, Albert
  • Nguyen, Truong Thao
  • Bautista Gomez, Leonardo
  • Takano, Ryousei
  • Badia Sala, Rosa Maria
  • Wahib, Mohamed
Deep Neural Network (DNN) frameworks use distributed training to enable faster time to convergence and alleviate memory capacity limitations when training large models and/or using high dimension inputs. With the steady increase in datasets and model sizes, model/hybrid parallelism is deemed to have an important role in the future of distributed training of DNNs. We analyze the compute, communication, and memory requirements of Convolutional Neural Networks (CNNs) to understand the trade-offs between different parallelism approaches on performance and scalability. We leverage our model-driven analysis to be the basis for an oracle utility which can help in detecting the limitations and bottlenecks of different parallelism approaches at scale. We evaluate the oracle on six parallelization strategies, with four CNN models and multiple datasets (2D and 3D), on up to 1024 GPUs. The results demonstrate that the oracle has an average accuracy of about 86.74% when compared to empirical results, and as high as 97.57% for data parallelism., The project that gave rise to these results received the support of a fellowship from the "la Caixa" Foundation (ID 100010434). The fellowship code is LCF/BQ/DI17/11620059. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713673. The Eurolab4HPC project has received funding from the European Union Horizon 2020 Framework Programme (H2020-EU.1.2.2. - FET Proactive) under grant agreement number 800962. This work was supported by JST, ACT-X Grant Number JPMJAX190C, Japan; by JST, PRESTO Grant Number JPMJPR20MA, Japan., Peer Reviewed, Postprint (author's final draft)