DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication
Contribución de congreso (ConferenceOutput)
2D-to-3D facial expression transfer
Trabajo presentado en la 24th International Conference on Pattern Recognition (ICPR), celebrada en Beijing (China), del 20 al 24 de agosto de 2018, Automatically changing the expression and physical features of a face from an input image is a topic that has been traditionally tackled in a 2D domain. In this paper, we bring this problem to 3D and propose a framework that given an input RGB video of a human face under a neutral expression, initially computes his/her 3D shape and then performs a transfer to a new and potentially non-observed expression. For this purpose, we parameterize the rest shape -obtained from standard factorization approaches over the input video- using a triangular mesh which is further clustered into larger macro-segments. The expression transfer problem is then posed as a direct mapping between this shape and a source shape, such as the blend shapes of an off-the-shelf 3D dataset of human facial expressions. The mapping is resolved to be geometrically consistent between 3D models by requiring points in specific regions to map on semantic equivalent regions. We validate the approach on several synthetic and real examples of input faces that largely differ from the source shapes, yielding very realistic expression transfers even in cases with topology changes, such as a synthetic video sequence of a single-eyed cyclops., This work is supported in part by the
Spanish Ministry of Science and Innovation under projects
FireDMMI TIN2014-56919-C3-2-R, BOSSS TIN2017-89723-
P, HuMoUR TIN2017-90086-R, and María de Maeztu Seal of
Excellence MDM-2016-0656, Peer reviewed
Project:
MINECO / Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 TIN2014-56919-C3-2-R
,
MINECO / Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 MDM-2016-0656
,
MINECO / Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 TIN2017-89723-P
,
MINECO / Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 TIN2017-90086-R
View at: http://hdl.handle.net/10261/179781