DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication
Artículo científico (JournalArticle)
Real-time 3D reconstruction of non-rigid shapes with a single moving camera
This paper describes a real-time sequential method to simultaneously recover the camera motion and the 3D shape of deformable objects from a calibrated monocular video. For this purpose, we consider the Navier-Cauchy equations used in 3D linear elasticity and solved by finite elements, to model the time-varying shape per frame. These equations are embedded in an extended Kalman filter, resulting in sequential Bayesian estimation approach. We represent the shape, with unknown material properties, as a combination of elastic elements whose nodal points correspond to salient points in the image. The global rigidity of the shape is encoded by a stiffness matrix, computed after assembling each of these elements. With this piecewise model, we can linearly relate the 3D displacements with the 3D acting forces that cause the object deformation, assumed to be normally distributed. While standard finite-element-method techniques require imposing boundary conditions to solve the resulting linear system, in this work we eliminate this requirement by modeling the compliance matrix with a generalized pseudoinverse that enforces a pre-fixed rank. Our framework also ensures surface continuity without the need for a post-processing step to stitch all the piecewise reconstructions into a global smooth shape. We present experimental results using both synthetic and real videos for different scenarios ranging from isometric to elastic deformations. We also show the consistency of the estimation with respect to 3D ground truth data, include several experiments assessing robustness against artifacts and finally, provide an experimental validation of our performance in real time at frame rate for small maps., This work has been partially supported by the Spanish Ministry of Science and Innovation under projects RT-SLAM DPI2015-67275-P, RobInstruct TIN2014-58178-R and Keratocono DPI2014-54981-R; by the FP7-SME-2013 606634 PopCorn project from the European Union FP7; by the ERA-net CHISTERA project VISEN PCIN-2013-047 and I-DRESS PCIN-2015-147; and by a scholarship FPU12/04886 awarded by the Spanish MECD., Peer reviewed
Project:
MINECO / Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 DPI2014-54981-R
,
EC / FP7 606634
,
MINECO / Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 TIN2014-58178-R
,
MINECO / Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 DPI2015-67275-P
,
MINECO / Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 PCIN-2013-047
,
MINECO / Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 PCIN-2015-147
View at: http://hdl.handle.net/10261/132981