Method of modeling objects to synthesize three-dimensional, photo-realistic animations

DETAILED DESCRIPTION The need remaining in the prior art is addressed by the present invention, which relates to a method for modeling three-dimensional objects and, more particularly, to modeling three-dimensional objects using a data-driven approach with separate three-dimensional image planes so as to synthesize three-dimensional, photo-realistic animations.
In accordance with the present invention, a data-driven approach is used, where a three-dimensional object, such as a talking person, is defined by a set of three-dimensional planes that approximate the shape and surrounding area of the object.
The object is then recorded on video and image recognition is applied to automatically extract bitmaps of each three-dimensional plane.
In the case of modeling a human face, the set of three-dimensional planes correspond to a set of pre-defined facial parts.
These bitmaps are then normalized and parameterized before being entered into a database.
For the synthesis of a human head, a text-to-speech synthesizer provides the audio track, as well as a phoneme string and trajectory that calculates motion for all the facial parts, including the whole head.
These trajectories provide the parameters for selecting the proper bitmaps from the database.
Smoothing and blending are applied to these `strings` of bitmaps to eliminate hard transitions and create a seamless animation for each facial part.
The result is a talking head that resembles very closely the person who was originally recorded.
It is an aspect of the present invention that the use of sample-based modeling from video images preserves a high level of detail in the appearance of the object.
For example, by recording real movements of a head and lips, and reusing them for the synthesis, a model is obtained that is able to produce realistic lip and head movements, as well as emotional expressions.
In defining the inventive modeling of a three-dimensional object, the flexibility of 3D models is combined with the realism of images.
A key problem with prior art sample-based techniques, as discussed above, is the ability to control the number of image samples that need to be recorded and stored