Method for extracting a three-dimensional model using appearance-based constrained structure from motion

DETAILED DESCRIPTION Provided is a computerized method for recovering 3-D models from a sequence of uncalibrated images with unknown point correspondences.
To that end, tracking, structure from motion with geometric constraints, and use of deformable 3-D models are integrated in a single framework.
The key to making the recovery method is the use of appearance-based model matching and refinement.
This appearance-based structure from motion approach is especially useful in recovering shapes of objects whose general structure is known but which may have little discernable texture in significant parts of their surfaces.
The method can be applied to 3-D face modeling from multiple images to create new 3-D faces for a synthetic talking head.
The talking head includes a collection of 3-D triangular facets, with nodes as vertices.
The model can be recovered even when the sequence of images are taken with a camera with unknown camera focal length and extrinsic parameters, i.
e.
, the camera pose is unknown.
In the general method, a three-dimensional model is extracted from a sequence of images including a reference image.
Each image in the sequence is registered with the reference image to determine image features.
The image features are used to recover structure and motion parameters using geometric constraints on a 3D wireframe mesh template.
A predicted appearance is generated for each image using the recovered structure and motion parameters, and each predicted appearance is then registered with the corresponding image.
The structure and motion recovery, appearance generation, and image registration steps are repeated until a selected termination condition is reached, for example, an average difference between the predicted appearances and the corresponding images is less than a predetermined threshold, or a fixed number of iterations has been performed.