Pasar al contenido principal

From perceptual modeling to semantic understanding of images and videos: methodology, algorithms and applications of egocentric vision

RYC-2017-22563

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de Promoción del Talento y su Empleabilidad en I+D+i
Subprograma Subprograma Estatal de Incorporación
Convocatoria Ramón y Cajal
Año convocatoria 2017
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario AGENCIA ESTATAL CONSEJO SUPERIOR DE INVESTIGACIONES CIENTIFICAS (CSIC)

Resultados relacionados

Resultados totales (Incluyendo duplicados): 1
Encontrada(s) 1 página(s)

DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Enhancing egocentric 3D pose estimation with third person views

  • Dhamanaskar, Ameya
  • Dimiccoli, Mariella
  • Corona, Enric
  • Pumarola, Albert
  • Moreno-Noguer, Francesc
We propose a novel approach to enhance the 3D body pose estimation of a person computed from videos captured from a single wearable camera. The main technical contribution consists of leveraging high-level features linking first- and third-views in a joint embedding space. To learn such embedding space we introduce First2Third-Pose, a new paired synchronized dataset of nearly 2000 videos depicting human activities captured from both first- and third-view perspectives. We explicitly consider spatial- and motion-domain features, combined using a semi-Siamese architecture trained in a self-supervised fashion. Experimental results demonstrate that the joint multi-view embedded space learned with our dataset is useful to extract discriminatory features from arbitrary single-view egocentric videos, with no need to perform any sort of domain adaptation or knowledge of camera parameters. An extensive evaluation demonstrates that we achieve significant improvement in egocentric 3D body pose estimation performance on two unconstrained datasets, over three supervised state-of-the-art approaches. The collected dataset and pre-trained model are available for research purposes., This work has been partially supported by projects PID2020-120049RB-I00 and PID2019-110977GA-I00 funded by MCIN/ AEI /10.13039/501100011033 and by the “European Union NextGenerationEU/PRTR”, as well as by grant RYC-2017-22563 funded by MCIN/ AEI /10.13039/501100011033 and by “ESF Investing in your future”, and network RED2018-102511-T funded by MCIN/ AEI., Peer reviewed