Credit: Disney Research A digitally created face can have the most realistic looking skin imaginable and still look fabricated to audiences if the eyes aren't quite right. But a new technique developed by Disney Research can capture the crucial, yet subtle details of the eyes with just a single facial scan, or even a single photo.
The method requires far less time and hardware than existing eye capture techniques because it employs a parametric model of the eye. The model can reproduce the variations in the size and shape of the eyeball, the spots, craters and banding of the colored iris and even the red veins of the white sclera.
The model can be used to automatically duplicate the details of an actor's eyes as captured in a scan or photo and can then be manipulated as needed to accommodate a story line, such as changing the dilation of the pupils to match lighting conditions, or increasing the size of veins to make the eyes look fatigued.
"The eyes are arguably the most important part of the face," said Markus Gross, vice president at Disney Research. "That's where we focus first when we look at someone and it's the eyes that convey emotions and foretell actions that are critical to storytelling. This new method of eye capture enables us to create highly realistic animations for films, games and medical applications and to do so with as little fuss as possible."
The research team will present their method July 24 at the ACM International Conference on Computer Graphics & Interactive Techniques (SIGGRAPH) in Anaheim, Calif.
After decades of research, facial capture technology is now so advanced that it can be difficult to distinguish between digital faces and real ones, said Thabo Beeler, senior research scientist at Disney Research. Relatively little attention has been given to eye capture, though previous work at Disney has produced a method to scan human eyes in very high quality.
The problem, Beeler said, is that the process requires actors to lie horizontally, heads immobilized by neck braces, while they manually hold their eyes open for dozens of photos over a 20-minute period.
"The physical burden of that approach is quite far from the single-shot face scanners that exist today, which are as easy as taking a single photo," he noted.
The parametric model that Beeler, Gross and their colleagues have created is based in large part on a database of 30 eyes that were captured in high resolution using that exhaustive process. The database provides details about the white sclera and the colored iris, while a separate model was used to represent variations in the size and shape of the eyeball itself.
Details of an actor's eye can be obtained with a single facial scan. The researchers developed a fitting algorithm that can then transfer these details to the eye model. Beeler said this approach is quite flexible and can be used with any traditional face capture setup. By limiting the parameters, it's even possible to fit the model to just a single photo of an actor, or even to an artistic painting.
"The results are very plausible eye reconstructions with realistic details," said Pascal Bérard, a Ph.D. student at Disney and ETH Zurich. "The model allows us to manipulate the captured data as it is fully parametric, such as changing the amount and appearance of sclera veins to simulate irritation or fatigue, or controlling the pupil size in response to synthetic illumination."
Explore further: Researchers invent system for 3-D reconstruction of sparse facial hair and skin
More information: "Lightweight Eye Capture Using a Parametric Model-Paper", s3-us-west-1.amazonaws.com/disneyresearch/wp-content/uploads/20160705182808/Lightweight-Eye-Capture-Using-a-Parametric-Model-Paper.pdf