As the cinematics lead for Call of Duty: Black Ops II, Adam Rosas has a tough job. Rosas had to use current-generation console technology to create convincing facial expressions and performances for the characters in the game such as Frank Woods. The end result are some highly realistic animated characters who have believable face-to-face conversations in the title’s movie-like sequences that take place in between gameplay scenes. (Via VentureBeat.com)
In a presentation at the Santa Monica headquarters of Treyarch, the Activision Blizzard game studio that made Black Ops II, Rosas described the process that the team used to create the cinematics in the game. As a division of the largest video game publisher, Treyarch spares no expense in getting the cinematics right since Black Ops II is part of the best-selling series of all time.
“The goal is to pick up natural motion,” Rosas said in an interview with GamesBeat.”This is not an artist’s interpretation of what the performance should be. We work with actors to bring emotion to a scene. The markers on the actor’s face are in positions so they capture muscle clusters like the mouth or the side of the eyes. It adds so much more life to what we are trying to bring in terms of our storytelling abilities.”
The team studied an actress reading three versions of a script. In the scene, the actress is alternately angry, sad, or controlled. They looked at the results of the test and then created a new system to capture the emotion from the infrared camera data.
The performance capture is converted into digital form and then automatically converted into an animated 3D image that the animators can work with. (That process is automatic now, but it used to take a week and a half). Rosas said the hardest thing to get right is the movement of a character’s mouth. Capturing the lip-synch is adequate. But fully capturing the movement of a person’s lips is very hard.
“When you look at the raw data, you see issues around the mouth,” Rosas said.
Rosas said that the team could use reflective make-up as created by Mova‘s motion-capture team. But he said Mova’s technology works only with faces, but the Treyarch team needed to capture not only an actor’s facial expressions, but their entire body movements as well. They also wanted to be able to take a performance and apply it to a character whose face was different from the actor’s.
“When you capture it all together, it looks more authentic,” Rosas said.
There are times when you’re speaking, for instance, when your lips become very thin. Other times, they become puffy or fat. The rest of the facial muscles move in tandem. Your nose may scrunch and your eyes may squint. The artists have to hand-draw the final parts of the lips to make sure they look realistic. That’s how the cinematics team can capture an emotional character scene up close.
Now the new scenes reveal subtle movements, such as a lip quivering, a nose sniffling or a brow furling.
The same motion-capture equipment worked for animating horses. In one scene, the soldiers have to hop onto horses to go after the bad guys. To create the cinematic, Rosas’ team had to capture a performance with an actor riding a real horse.
“It was a very scary proposition,” Rosas said. “Not many game productions have captured a horse.”
But the people were still the toughest problem.
“In terms of emotional connections with the player, we feel we have it now,” Rosas said. “We’re very excited about where we are going with this.”