Skip to main content

ANALISIS GEOMETRICO Y SEMANTICO DE VIDEO

PGC2018-098625-B-I00

•
Funding agency name Agencia Estatal de Investigación
Funding agency acronym AEI
Program Programa Estatal de Generación de Conocimiento y Fortalecimiento Científico y Tecnológico del Sistema de I+D+i
Subprogram Subprograma Estatal de Generación de Conocimiento
Call Proyectos de I+D (Excelencia / Generación de Conocimiento)
Call year 2018
State Plan 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Beneficiary centre UNIVERSITAT POMPEU FABRA CCT

Resultados relacionados

Resultados totales (Incluyendo duplicados): 6
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Self-supervised small soccer player detection and tracking

  • Hurault, Samuel
  • Ballester, Coloma
  • Haro Ortega, Gloria
Comunicació presentada al MMSports '20: The 3rd International Workshop on Multimedia Content Analysis in Sports, celebrat el 16 d'octubre de 2020 de manera virtual., In a soccer game, the information provided by detecting and tracking brings crucial clues to further analyze and understand some tactical aspects of the game, including individual and team actions. State-of-the-art tracking algorithms achieve impressive results in scenarios on which they have been trained for, but they fail in challenging ones such as soccer games. This is frequently due to the player small relative size and the similar appearance among players of the same team. Although a straightforward solution would be to retrain these models by using a more specific dataset, the lack of such publicly available annotated datasets entails searching for other effective solutions. In this work, we propose a self-supervised pipeline which is able to detect and track low-resolution soccer players under different recording conditions without any need of ground-truth data. Extensive quantitative and qualitative experimental results are presented evaluating its performance. We also present a comparison to several state-of-the-art methods showing that both the proposed detector and the proposed tracker achieve top-tier results, in particular in the presence of small players., SH acknowledges support by ENS Paris-Saclay. CB and GH acknowledge support by MICINN/FEDER UE project, ref. PGC2018-098625- B-I00, H2020-MSCA-RISE-2017 project, ref. 777826 NoMADS, and RED2018-102511-T.
Repositorio Digital de la UPF
Publication Contribución de congreso (ConferenceOutput)

Semantic image completion through an adversarial strategy

  • Vitoria, Patricia
  • Sintes Marcos, Joan
  • Ballester, Coloma
Comunicació presentada a: VISIGRAPP 2019: Computer Vision, Imaging and Computer Graphics Theory and Applications. 14th International Joint Conference, celebrat del 25 al 27 de febrer de 2019 a Praga, República Txeca., Image completion or image inpainting is the task of filling in missing regions of an image. When those areas are large and the missing information is unique such that the information and redundancy available in the image is not useful to guide the completion, the task becomes even more challenging. This paper proposes an automatic semantic inpainting method able to reconstruct corrupted information of an image by semantically interpreting the image itself. It is based on an adversarial strategy followed by an energy-based completion algorithm. First, the data latent space is learned by training a modified Wasserstein generative adversarial network. Second, the learned semantic information is combined with a novel optimization loss able to recover missing regions conditioned by the available information. Moreover, we present an application in the context of face inpainting, where our method is used to generate a new face by integrating desired facial attributes or expressions from a reference face. This is achieved by slightly modifying the objective energy. Quantitative and qualitative top-tier results show the power and realism of the presented method., The authors acknowledge partial support by MICINN/FEDER UE project, reference PGC2018-098625-B-I00 and by H2020-MSCA-RISE-2017 project, reference 777826 NoMADS. We also thank NVIDIA for the Quadro P6000 GPU donation.
Repositorio Digital de la UPF
Publication Artículo científico (JournalArticle)

Conditioned source separation for musical instrument performances

  • Slizovskaia, Olga
  • Haro Ortega, Gloria
  • Gómez Gutiérrez, Emilia, 1975-
In music source separation, the number of sources may vary for each piece and some of the sources may belong to the same family of instruments, thus sharing timbral characteristics and making the sources more correlated. This leads to additional challenges in the source separation problem. This paper proposes a source separation method for multiple musical instruments sounding simultaneously and explores how much additional information apart from the audio stream can lift the quality of source separation. We explore conditioning techniques at different levels of a primary source separation network and utilize two extra modalities of data, namely presence or absence of instruments in the mixture, and the corresponding video stream data., This work was funded in part by ERC Innovation Programme (grant 770376, TROMPA); Spanish Ministry of Economy and Competitiveness under the Mar´ıa de Maeztu Units of Excellence Program (MDM-2015-0502) and the Social European Funds; the MICINN/FEDER UE project with reference PGC2018-098625-B-I00; and the H2020-MSCARISE-2017 project with reference 777826 NoMADS. We gratefully acknowledge NVIDIA for the donation of GPUs used for the experiments.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Self-supervised small soccer player detection and tracking

  • Hurault, Samuel
  • Ballester, Coloma
  • Haro Ortega, Gloria
Comunicació presentada al MMSports '20: The 3rd International Workshop on Multimedia Content Analysis in Sports, celebrat el 16 d'octubre de 2020 de manera virtual., In a soccer game, the information provided by detecting and tracking brings crucial clues to further analyze and understand some tactical aspects of the game, including individual and team actions. State-of-the-art tracking algorithms achieve impressive results in scenarios on which they have been trained for, but they fail in challenging ones such as soccer games. This is frequently due to the player small relative size and the similar appearance among players of the same team. Although a straightforward solution would be to retrain these models by using a more specific dataset, the lack of such publicly available annotated datasets entails searching for other effective solutions. In this work, we propose a self-supervised pipeline which is able to detect and track low-resolution soccer players under different recording conditions without any need of ground-truth data. Extensive quantitative and qualitative experimental results are presented evaluating its performance. We also present a comparison to several state-of-the-art methods showing that both the proposed detector and the proposed tracker achieve top-tier results, in particular in the presence of small players., SH acknowledges support by ENS Paris-Saclay. CB and GH acknowledge support by MICINN/FEDER UE project, ref. PGC2018-098625- B-I00, H2020-MSCA-RISE-2017 project, ref. 777826 NoMADS, and RED2018-102511-T.
Recercat. Dipósit de la Recerca de Catalunya
Publication Contribución de congreso (ConferenceOutput)

Semantic image completion through an adversarial strategy

  • Vitoria, Patricia
  • Sintes Marcos, Joan
  • Ballester, Coloma
Comunicació presentada a: VISIGRAPP 2019: Computer Vision, Imaging and Computer Graphics Theory and Applications. 14th International Joint Conference, celebrat del 25 al 27 de febrer de 2019 a Praga, República Txeca., Image completion or image inpainting is the task of filling in missing regions of an image. When those areas are large and the missing information is unique such that the information and redundancy available in the image is not useful to guide the completion, the task becomes even more challenging. This paper proposes an automatic semantic inpainting method able to reconstruct corrupted information of an image by semantically interpreting the image itself. It is based on an adversarial strategy followed by an energy-based completion algorithm. First, the data latent space is learned by training a modified Wasserstein generative adversarial network. Second, the learned semantic information is combined with a novel optimization loss able to recover missing regions conditioned by the available information. Moreover, we present an application in the context of face inpainting, where our method is used to generate a new face by integrating desired facial attributes or expressions from a reference face. This is achieved by slightly modifying the objective energy. Quantitative and qualitative top-tier results show the power and realism of the presented method., The authors acknowledge partial support by MICINN/FEDER UE project, reference PGC2018-098625-B-I00 and by H2020-MSCA-RISE-2017 project, reference 777826 NoMADS. We also thank NVIDIA for the Quadro P6000 GPU donation.
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Conditioned source separation for musical instrument performances

  • Slizovskaia, Olga
  • Haro Ortega, Gloria
  • Gómez Gutiérrez, Emilia, 1975-
In music source separation, the number of sources may vary for each piece and some of the sources may belong to the same family of instruments, thus sharing timbral characteristics and making the sources more correlated. This leads to additional challenges in the source separation problem. This paper proposes a source separation method for multiple musical instruments sounding simultaneously and explores how much additional information apart from the audio stream can lift the quality of source separation. We explore conditioning techniques at different levels of a primary source separation network and utilize two extra modalities of data, namely presence or absence of instruments in the mixture, and the corresponding video stream data., This work was funded in part by ERC Innovation Programme (grant 770376, TROMPA); Spanish Ministry of Economy and Competitiveness under the Mar´ıa de Maeztu Units of Excellence Program (MDM-2015-0502) and the Social European Funds; the MICINN/FEDER UE project with reference PGC2018-098625-B-I00; and the H2020-MSCARISE-2017 project with reference 777826 NoMADS. We gratefully acknowledge NVIDIA for the donation of GPUs used for the experiments.