Pasar al contenido principal

MODELOS Y TECNICAS COMPUTACIONALES PARA EL ANALISIS Y PROCESAMIENTO DE VIDEO

TIN2015-70410-C2-1-R

•
Nombre agencia financiadora Ministerio de Economía y Competitividad
Acrónimo agencia financiadora MINECO
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos I+D+i Retos de Investigación
Año convocatoria 2015
Plan Estatal 2013-2016 Plan Estatal de Investigación Científica y Técnica y de Innovación
Unidad de gestión Dirección General de Investigación Científica y Técnica
Centro beneficiario UNIVERSITAT POMPEU FABRA CCT
Identificador persistente http://dx.doi.org/10.13039/501100003329

Resultados relacionados

Resultados totales (Incluyendo duplicados): 18
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

End-to-end sound source separation conditioned on instrument labels

  • Slizovskaia, Olga
  • Kim, Leo
  • Haro Ortega, Gloria
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing celebrat de 12 al 17 de maig de 2019 a Brighton, Regne Unit., Can we perform an end-to-end music source separation with a variable number of sources using a deep learning model? This paper presents an extension of the Wave-UNet [1] model which allows end-to-end monaural source separation with a non-fixed number of sources. Furthermore, we propose multiplicative conditioning with instrument labels at the bottleneck of the Wave-U-Net and show its effect on the separation results. This approach can be further extended to other types of conditioning such as audio-visual source separation and score-informed source separation., This work has received funding from the Maria de Maeztu Programme (MDM-2015-0502), ERC Innovation Programme (grant 770376, TROMPA), and MINECO/FEDER UE project (TIN2015-70410-C2-1-R).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Musical instrument recognition in user-generated videos using a multimodal convolutional neural network architecture

  • Slizovskaia, Olga
  • Gómez Gutiérrez, Emilia, 1975-
  • Haro Ortega, Gloria
Comunicació presentada a la International Conference on Multimedia Retrieval celebrada del 6 al 9 de juny de 2017 a Bucarest, Romania., This paper presents a method for recognizing musical instruments in user-generated videos. Musical instrument recognition from music signals is a well-known task in the music information retrieval (MIR) field, where current approaches rely on the analysis of the good-quality audio material. This work addresses a real-world scenario with several research challenges, i.e. the analysis of user-generated videos that are varied in terms of recording conditions and quality and may contain multiple instruments sounding simultaneously and background noise. Our approach does not only focus on the analysis of audio information, but we exploit the multimodal information embedded in the audio and visual domains. In order to do so, we develop a Convolutional Neural Network (CNN) architecture which combines learned representations from both modalities at a late fusion stage. Our approach is trained and evaluated on two large-scale video datasets: YouTube-8M and FCVID. The proposed architectures demonstrate state-of-the-art results in audio and video object recognition, provide additional robustness to missing modalities, and remains computationally cheap to train., is work is partly supported by the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502), the CASAS Spanish research project (TIN2015-70816-R), and project TIN2015-70410-C2- 1-R (MINECO/FEDER, UE). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan X GPU used for this research.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Motion inpainting by an image-based geodesic AMLE method

  • Oliver, Maria
  • Raad, Lara
  • Ballester, Coloma
  • Haro Ortega, Gloria
Comunicació presentada al congrés 25th IEEE International Conference on Image Processing (ICIP) celebrat del 7 al 10 d'octubre de 2018 a Atenes, Grècia., This work presents an automatic method for optical flow inpainting. Given a video, each frame domain is endowed with a Riemannian metric based on the video pixel values. The missing optical flow is recovered by solving the Absolutely Minimizing Lipschitz Extension (AMLE) partial differential equation on the Riemannian manifold. An efficient numerical algorithm is proposed using eikonal operators for nonlinear elliptic partial differential equations on a finite graph. The choice of the metric is discussed and the method is applied to optical flow inpainting and sparse-to-dense optical flow estimation, achieving top-tier performance in terms of End-Point-Error (EPE)., This work is partially supported by MINECO/FEDER UE project, with reference TIN2015-70410-C2-1-R and by RISE project NoMADS.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Semantic image inpainting through improved Wasserstein generative adversarial networks

  • Vitoria, Patricia
  • Sintes Marcos, Joan
  • Ballester, Coloma
Comunicació presentada a: VISAPP 2019, celebrada a Praga, República Txeca, del 25 al 27 de febrer de 2019, Image inpainting is the task of filling-in missing regions of a damaged or incomplete image. In this work we tackle this problem not only by using the available visual data but also by incorporating image semantics through the use of generative models. Our contribution is twofold: First, we learn a data latent space by training an improved version of the Wasserstein generative adversarial network, for which we incorporate a new generator and discriminator architecture. Second, the learned semantic information is combined with a new optimization loss for inpainting whose minimization infers the missing content conditioned by the available data. It takes into account powerful contextual and perceptual content inherent in the image itself. The benefits include the ability to recover large regions by accumulating semantic information even it is not fully present in the damaged image. Experiments show that the presented method obtains qualitative and quantitative top-tier results in different experimental situations and also achieves accurate photo-realism comparable to state-of-the-art works., The authors acknowledge partial support by MINECO/FEDER UE project, reference TIN2015-70410-C2-1 and by H2020-MSCA-RISE-2017 project, reference 777826 NoMADS.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

A multiclass anisotropic Mumford-Shah functional for segmentation of d-dimensional vectorial images

  • Garamendi, Juan Francisco
  • Schiavi, Emanuele
Comunicació presentada a la 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2017), celebrada del 27 de febrer l'1 de març de 2017 a Porto, Portugal., We present a general model for multi-class segmentation of multi-channel digital images. It is based on the minimization of an anisotropic version of the Mumford-Shah energy functional in the class of piecewise constant functions. In the framework of geometric measure theory we use the concept of common interphases between regions (classes) and the value of the jump discontinuities of the (weak) solution between adjacent regions in order to define a minimal partition energy functional. The resulting problem is non-smooth and non-convex. Non-smoothness is dealt with highlighting the relationship of the proposed model with the well known Rudin, Osher and Fatemi model for image denoising when piecewise constant solutions (i.e partitions) are considered. Non-convexity is tackled with an optimal threshold of the ROF solution which we which generalize to multi-channel images through a probabilistic clustering. The optimal solution is then computed with a fixed point iteration. The result ing algorithm is described and results are presented showing the successful application of the method to Light Field (LF) images., The authors acknowledge partial support by TIN2015-70410-C2-1-R (MINECO/FEDER, UE) and by GRC reference 2014 SGR 1301, Generalitat de Catalunya.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Exemplar-based image inpainting using an affine invariant similarity measure

  • Fedorov, Vadim
  • Arias Martínez, Pablo
  • Facciolo Furlan, Gabriele
  • Ballester, Coloma
Comunicació presentada a 11th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2016), celebrat del 27 al 29 de febrer de 2016 a Roma, Itàlia., Patch-based approaches are used in state-of-the-art methods for image inpainting. This paper presents a new method for exemplar-based image inpainting using transformed patches. The transformation is determined for each patch in a fully automatic way from a surrounding texture content. We build upon a recent affine invariant patch similarity measure that performs an appropriate patch comparison by automatically adapting the size and shape of the patches. As a consequence, it intrinsically extends the set of available source patches to copy information from. We incorporate this measure into a variational formulation for inpainting and present a numerical algorithm for optimizing it. We show that our method can be applied to complete a perspectively distorted texture as well as to automatically inpaint one view of a scene using other view of the same scene as a source. We present experimental results both for gray and color images, and a comparison with some exemplar-based image inpainting methods., The first, second and fourth authors acknowledge partial support by the MINECO/FEDER project with reference TIN2015-70410-C2-1-R , the MICINN project with reference MTM2012- 30772, and by GRC reference 2014 SGR 1301, Generalitat de Catalunya. The second and third authors were partly founded by the Centre National dEtudes Spatiales (CNES, MISS Project), BPIFrance and R´egion Ile de France, in the framework of the FUI 18 Plein Phare project, the European Research Council (advanced grant Twelve Labours n246961), the Office of Naval research (ONR grant N00014-14-1-0023), and ANRDGA project ANR-12-ASTR-0035.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

From occlusion to global depth order, a monocular approach

  • Rezaeirowshan, Babak
  • Ballester, Coloma
  • Haro Ortega, Gloria
Comunicació presentada al 11th International Joint Conference (VISIGRAPP), celebrat del 27 al 29 de febrer de 2016 a Roma, Itàlia., Estimating 3D structure of the scene from a single image remains a challenging problem in computer vision. This paper proposes a novel approach to obtain a global depth order of objects by incorporating monocular perceptual cues such as T-junctions and object boundary convexity, which are local indicators of occlusions, together with physical cues, namely ground contact points. The proposed combination of these local cues complement each other and creates a more thorough partial depth order relationship. The different partial orders are then robustly aggregated using a Markov random chain approximation to obtain the most plausible global depth order. Experiments show that the proposed method excels in comparison to state of the art methods., The authors acknowledge partial support by the MINECO/FEDER project with reference TIN2015-70410-C2-1-R, the MICINN project with reference MTM2012-30772, and by GRC reference 2014 SGR 1301, Generalitat de Catalunya.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Joint large displacement scene flow and occlusion variational estimation

  • Palomares, Roberto P.
  • Haro Ortega, Gloria
  • Ballester, Coloma
Comunicació presentada al congrés International Conference on Computer Vision Theory and Applications (VISSAP) celebrat a Porto (Portugal) del 27 de febrer a l'1 de març de 2017., This paper presents a novel variational approach for the joint estimation of scene flow and occlusions. Our method does not assume that a depth sensor is available. Instead, we use a stereo sequence and exploit the fact that points that are occluded in time, might be visible from the other view and thus the 3D geometry can be densely reinforced in an appropriate manner through a simultaneous motion occlusion characterization. Moreover, large displacements are correctly captured thanks to an optimization strategy that uses a set of sparse image correspondences to guide the minimization process. We include qualitative and quantitative experimental results on several datasets illustrating that both proposals help to improve the baseline results., The authors acknowledge partial support by TIN2015-70410-C2-1-R (MINECO/FEDER, UE) and by GRC reference 2014 SGR 1301, Generalitat de Catalunya.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Visual music transcription of clarinet video recordings trained with audio-based labelled data

  • Gómez Gutiérrez, Emilia, 1975-
  • Arias Martínez, Pablo
  • Zinemanas, Pablo
  • Haro Ortega, Gloria
Comunicació presentada a la 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), celebrada els dies 22 a 29 d'octubre de 2017 a Venècia, Itàlia., Automatic transcription is a well-known task in the music information retrieval (MIR) domain, and consists on the computation of a symbolic music representation (e.g. MIDI) from an audio recording. In this work, we address the automatic transcription of video recordings when the audio modality is missing or it does not have enough quality, and thus analyze the visual information. We focus on the clarinet which is played by opening/closing a set of holes and keys. We propose a method for automatic visual note estimation by detecting the fingertips of the player and measuring their displacement with respect to the holes and keys of the clarinet. To this aim, we track the clarinet and determine its position on every frame. The relative positions of the fingertips are used as features of a machine learning algorithm trained for note pitch classification. For that purpose, a dataset is built in a semiautomatic way by estimating pitch information from audio signals in an existing collection of 4.5 hours of video recordings from six different songs performed by nine different players. Our results confirm the difficulty of performing visual vs audio automatic transcription mainly due to motion blur and occlusions that cannot be solved with a single view., This work is partly supported by the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502); the Spanish research projects CASAS (TIN2015-70816-R, MINECO/FEDER, UE) and MOTECAVI (TIN2015-70410-C2-1-R, MINECO/FEDER, UE); BPIFrance and Région Île de France in the framework of the FUI 18 Plein Phare project; the Office of Naval research by grant N00014-17-1-2552, ANR-DGA project ANR-12-ASTR-0035; and ANR-14-CE27-001 (MIRIAM).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Joint large displacement scene flow and occlusion variational estimation

  • Palomares, Roberto P.
  • Haro Ortega, Gloria
  • Ballester, Coloma
Comunicació presentada al congrés International Conference on Computer Vision Theory and Applications (VISSAP) celebrat a Porto (Portugal) del 27 de febrer a l'1 de març de 2017., This paper presents a novel variational approach for the joint estimation of scene flow and occlusions. Our method does not assume that a depth sensor is available. Instead, we use a stereo sequence and exploit the fact that points that are occluded in time, might be visible from the other view and thus the 3D geometry can be densely reinforced in an appropriate manner through a simultaneous motion occlusion characterization. Moreover, large displacements are correctly captured thanks to an optimization strategy that uses a set of sparse image correspondences to guide the minimization process. We include qualitative and quantitative experimental results on several datasets illustrating that both proposals help to improve the baseline results., The authors acknowledge partial support by TIN2015-70410-C2-1-R (MINECO/FEDER, UE) and by GRC reference 2014 SGR 1301, Generalitat de Catalunya.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Exemplar-based image inpainting using an affine invariant similarity measure

  • Fedorov, Vadim
  • Arias Martínez, Pablo
  • Facciolo Furlan, Gabriele
  • Ballester, Coloma
Comunicació presentada a 11th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2016), celebrat del 27 al 29 de febrer de 2016 a Roma, Itàlia., Patch-based approaches are used in state-of-the-art methods for image inpainting. This paper presents a new method for exemplar-based image inpainting using transformed patches. The transformation is determined for each patch in a fully automatic way from a surrounding texture content. We build upon a recent affine invariant patch similarity measure that performs an appropriate patch comparison by automatically adapting the size and shape of the patches. As a consequence, it intrinsically extends the set of available source patches to copy information from. We incorporate this measure into a variational formulation for inpainting and present a numerical algorithm for optimizing it. We show that our method can be applied to complete a perspectively distorted texture as well as to automatically inpaint one view of a scene using other view of the same scene as a source. We present experimental results both for gray and color images, and a comparison with some exemplar-based image inpainting methods., The first, second and fourth authors acknowledge partial support by the MINECO/FEDER project with reference TIN2015-70410-C2-1-R , the MICINN project with reference MTM2012- 30772, and by GRC reference 2014 SGR 1301, Generalitat de Catalunya. The second and third authors were partly founded by the Centre National dEtudes Spatiales (CNES, MISS Project), BPIFrance and R´egion Ile de France, in the framework of the FUI 18 Plein Phare project, the European Research Council (advanced grant Twelve Labours n246961), the Office of Naval research (ONR grant N00014-14-1-0023), and ANRDGA project ANR-12-ASTR-0035.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

A multiclass anisotropic Mumford-Shah functional for segmentation of d-dimensional vectorial images

  • Garamendi, Juan Francisco
  • Schiavi, Emanuele
Comunicació presentada a la 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2017), celebrada del 27 de febrer l'1 de març de 2017 a Porto, Portugal., We present a general model for multi-class segmentation of multi-channel digital images. It is based on the minimization of an anisotropic version of the Mumford-Shah energy functional in the class of piecewise constant functions. In the framework of geometric measure theory we use the concept of common interphases between regions (classes) and the value of the jump discontinuities of the (weak) solution between adjacent regions in order to define a minimal partition energy functional. The resulting problem is non-smooth and non-convex. Non-smoothness is dealt with highlighting the relationship of the proposed model with the well known Rudin, Osher and Fatemi model for image denoising when piecewise constant solutions (i.e partitions) are considered. Non-convexity is tackled with an optimal threshold of the ROF solution which we which generalize to multi-channel images through a probabilistic clustering. The optimal solution is then computed with a fixed point iteration. The result ing algorithm is described and results are presented showing the successful application of the method to Light Field (LF) images., The authors acknowledge partial support by TIN2015-70410-C2-1-R (MINECO/FEDER, UE) and by GRC reference 2014 SGR 1301, Generalitat de Catalunya.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Semantic image inpainting through improved Wasserstein generative adversarial networks

  • Vitoria, Patricia
  • Sintes Marcos, Joan
  • Ballester, Coloma
Comunicació presentada a: VISAPP 2019, celebrada a Praga, República Txeca, del 25 al 27 de febrer de 2019, Image inpainting is the task of filling-in missing regions of a damaged or incomplete image. In this work we tackle this problem not only by using the available visual data but also by incorporating image semantics through the use of generative models. Our contribution is twofold: First, we learn a data latent space by training an improved version of the Wasserstein generative adversarial network, for which we incorporate a new generator and discriminator architecture. Second, the learned semantic information is combined with a new optimization loss for inpainting whose minimization infers the missing content conditioned by the available data. It takes into account powerful contextual and perceptual content inherent in the image itself. The benefits include the ability to recover large regions by accumulating semantic information even it is not fully present in the damaged image. Experiments show that the presented method obtains qualitative and quantitative top-tier results in different experimental situations and also achieves accurate photo-realism comparable to state-of-the-art works., The authors acknowledge partial support by MINECO/FEDER UE project, reference TIN2015-70410-C2-1 and by H2020-MSCA-RISE-2017 project, reference 777826 NoMADS.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Visual music transcription of clarinet video recordings trained with audio-based labelled data

  • Gómez Gutiérrez, Emilia, 1975-
  • Arias Martínez, Pablo
  • Zinemanas, Pablo
  • Haro Ortega, Gloria
Comunicació presentada a la 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), celebrada els dies 22 a 29 d'octubre de 2017 a Venècia, Itàlia., Automatic transcription is a well-known task in the music information retrieval (MIR) domain, and consists on the computation of a symbolic music representation (e.g. MIDI) from an audio recording. In this work, we address the automatic transcription of video recordings when the audio modality is missing or it does not have enough quality, and thus analyze the visual information. We focus on the clarinet which is played by opening/closing a set of holes and keys. We propose a method for automatic visual note estimation by detecting the fingertips of the player and measuring their displacement with respect to the holes and keys of the clarinet. To this aim, we track the clarinet and determine its position on every frame. The relative positions of the fingertips are used as features of a machine learning algorithm trained for note pitch classification. For that purpose, a dataset is built in a semiautomatic way by estimating pitch information from audio signals in an existing collection of 4.5 hours of video recordings from six different songs performed by nine different players. Our results confirm the difficulty of performing visual vs audio automatic transcription mainly due to motion blur and occlusions that cannot be solved with a single view., This work is partly supported by the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502); the Spanish research projects CASAS (TIN2015-70816-R, MINECO/FEDER, UE) and MOTECAVI (TIN2015-70410-C2-1-R, MINECO/FEDER, UE); BPIFrance and Région Île de France in the framework of the FUI 18 Plein Phare project; the Office of Naval research by grant N00014-17-1-2552, ANR-DGA project ANR-12-ASTR-0035; and ANR-14-CE27-001 (MIRIAM).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

From occlusion to global depth order, a monocular approach

  • Rezaeirowshan, Babak
  • Ballester, Coloma
  • Haro Ortega, Gloria
Comunicació presentada al 11th International Joint Conference (VISIGRAPP), celebrat del 27 al 29 de febrer de 2016 a Roma, Itàlia., Estimating 3D structure of the scene from a single image remains a challenging problem in computer vision. This paper proposes a novel approach to obtain a global depth order of objects by incorporating monocular perceptual cues such as T-junctions and object boundary convexity, which are local indicators of occlusions, together with physical cues, namely ground contact points. The proposed combination of these local cues complement each other and creates a more thorough partial depth order relationship. The different partial orders are then robustly aggregated using a Markov random chain approximation to obtain the most plausible global depth order. Experiments show that the proposed method excels in comparison to state of the art methods., The authors acknowledge partial support by the MINECO/FEDER project with reference TIN2015-70410-C2-1-R, the MICINN project with reference MTM2012-30772, and by GRC reference 2014 SGR 1301, Generalitat de Catalunya.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

End-to-end sound source separation conditioned on instrument labels

  • Slizovskaia, Olga
  • Kim, Leo
  • Haro Ortega, Gloria
  • Gómez Gutiérrez, Emilia, 1975-
Comunicació presentada a: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing celebrat de 12 al 17 de maig de 2019 a Brighton, Regne Unit., Can we perform an end-to-end music source separation with a variable number of sources using a deep learning model? This paper presents an extension of the Wave-UNet [1] model which allows end-to-end monaural source separation with a non-fixed number of sources. Furthermore, we propose multiplicative conditioning with instrument labels at the bottleneck of the Wave-U-Net and show its effect on the separation results. This approach can be further extended to other types of conditioning such as audio-visual source separation and score-informed source separation., This work has received funding from the Maria de Maeztu Programme (MDM-2015-0502), ERC Innovation Programme (grant 770376, TROMPA), and MINECO/FEDER UE project (TIN2015-70410-C2-1-R).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Musical instrument recognition in user-generated videos using a multimodal convolutional neural network architecture

  • Slizovskaia, Olga
  • Gómez Gutiérrez, Emilia, 1975-
  • Haro Ortega, Gloria
Comunicació presentada a la International Conference on Multimedia Retrieval celebrada del 6 al 9 de juny de 2017 a Bucarest, Romania., This paper presents a method for recognizing musical instruments in user-generated videos. Musical instrument recognition from music signals is a well-known task in the music information retrieval (MIR) field, where current approaches rely on the analysis of the good-quality audio material. This work addresses a real-world scenario with several research challenges, i.e. the analysis of user-generated videos that are varied in terms of recording conditions and quality and may contain multiple instruments sounding simultaneously and background noise. Our approach does not only focus on the analysis of audio information, but we exploit the multimodal information embedded in the audio and visual domains. In order to do so, we develop a Convolutional Neural Network (CNN) architecture which combines learned representations from both modalities at a late fusion stage. Our approach is trained and evaluated on two large-scale video datasets: YouTube-8M and FCVID. The proposed architectures demonstrate state-of-the-art results in audio and video object recognition, provide additional robustness to missing modalities, and remains computationally cheap to train., is work is partly supported by the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015-0502), the CASAS Spanish research project (TIN2015-70816-R), and project TIN2015-70410-C2- 1-R (MINECO/FEDER, UE). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan X GPU used for this research.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Motion inpainting by an image-based geodesic AMLE method

  • Oliver, Maria
  • Raad, Lara
  • Ballester, Coloma
  • Haro Ortega, Gloria
Comunicació presentada al congrés 25th IEEE International Conference on Image Processing (ICIP) celebrat del 7 al 10 d'octubre de 2018 a Atenes, Grècia., This work presents an automatic method for optical flow inpainting. Given a video, each frame domain is endowed with a Riemannian metric based on the video pixel values. The missing optical flow is recovered by solving the Absolutely Minimizing Lipschitz Extension (AMLE) partial differential equation on the Riemannian manifold. An efficient numerical algorithm is proposed using eikonal operators for nonlinear elliptic partial differential equations on a finite graph. The choice of the metric is discussed and the method is applied to optical flow inpainting and sparse-to-dense optical flow estimation, achieving top-tier performance in terms of End-Point-Error (EPE)., This work is partially supported by MINECO/FEDER UE project, with reference TIN2015-70410-C2-1-R and by RISE project NoMADS.