Digital Doubles of real actors is a holy-grail of the entertainment industry from film production to interactive games. Film production of computer generated characters has achieved photo-realistic emotionally believable performance, but requires highly skilled and time-consuming manual animation. Photo-realistic animation of real people remains a challenging problem because of the importance of subtle facial detail in conveying and interpreting emotion.
CVSSP research in 3D video capture of facial performance has made possible real-time acquisition of high-definition facial shape and appearance of actor performance. Raw 3D video capture is unstructured and and can not be directly used in conventional computer graphics pipeline where artistic control is of central importance for character interaction and photo-realistic rendering.
Research, led by Prof. Adrian Hilton, supported by the Royal Society and EPSRC in collaboration with Framestore, is investigating methods for integrating 3D video capture of facial performance with visual-effects production pipelines to allow the creation of highly realistic digital doubles. This research has introduced methods for high-resolution 3D facial capture (wrinkles, pores) and techniques for temporal alignment of non-rigid mesh sequences to obtain a structured representation suitable for use in animation production.