Technology
At the core of aPeerance
technology is FaceScript™, a descriptive system for specifying both the
features and state of a character's face. In the same way that PostScript™
describes the layout of a page through an understanding of the standard
elements of a document, FaceScript™ uses knowledge of the human face,
both anatomical and behavioral, to describe the type and expression in
a compact representation. This description can be used both as an authoring
scheme to describe the overall appearance of a character and as a real-time
animation system to change the expression and move the lips.
FaceScript™ is the result of more than a decade of academic research and
continued refinement in various applications, including cinema-quality
animations as well as interactive entertainment.
Authoring:
The essence of FaceScript™ is a set of numerical parameters, each of which
controls some aspect of a character's face. These parameters can be varied
independently to modify specific features of the face (e.g. cheek-bone
prominence, forehead height, jaw-width, etc.). This authoring paradigm
is highly flexible, allowing a wide range of applications. The entire
set of parameters can be exposed individually to an off-line character
designer for full authoring control or a sub-set of these parameters with
constraints can be presented to an end user for customization and personalization.
Higher-level constructs can be imposed on the basic parameter scheme by
combining low-level parameters to create application-specific descriptive
elements. For example a user could modify the character's appearance from
"sophisticated" to "silly" with a single control that simultaneously modifies
eye separation, forehead height, nose scale, etc.
The encapsulation of a face's features into a defined set of parameters
makes it possible to "search" for a desired character, rather than design
one. By treating each possible face as a vector in a vast multi-dimensional
space, it is possible to talk about similar faces as being "near-by" one
another. By "navigating" through this space, a user can "find" a character
that they recognize as appealing, as opposed to creating one from scratch.
This approach was employed in FaceLift™, a character-authoring tool for
Electronic Arts' most popular game, The Sims™. FaceLift™ presents the
user with a character and a set of its near-by neighbors in "face space".
The user chooses one of the neighbors in some desired "direction" (i.e.
more pirate-like or more grandmother-like, etc.) and then sees some of
the faces surrounding their new choice. By narrowing the range of neighboring
faces as the user converges on the character of choice, he or she eventually
arrives at a desired character without ever directly modifying any of
the low-level parameters.
Fourteen distinct
but related Sims™ faces that were encountered through a session of navigating
through "face space" using FaceLift™.
Animation:
Among the parameters in the FaceScript™ set are those representing more
transient states (i.e. jaw-drop, eye-lid closure, eyebrow lift, etc.).
By modifying these in real-time, a character's face can be animated with
a high degree of articulation with a minimum level of specification. Just
as in the earlier example, where characteristic parameters can be combined
into higher-level constructs, the more transient parameters can be combined
in real-time to express a change of emotional state.
This figure shows two characters with the same "fear" expression applied
to each. Since the parameters controlling the expression are independent
of those creating the character type, the same set of "fear deltas" creates
the impression of fear that is recognizable, but unique to each face.
Note particularly how the asymmetries inherent in the bottom character
are preserved while undergoing the fear transformation.
Another example of animating groups of transient parameters in real-time
is driving the mouth parameters from an audio source to perform real-time
lip-synching. Because of the variety of parameters to modify, the quality
of this voice-driven animation is limited only by the sophistication of
the voice-stream analysis. For example, eye-widening can be linked to
increased emphasis or head-tilting to rising inflection. Since the representation
of these voice responses is extremely compact (i.e. small sets of parametric
changes), these can be sent over-the-wire to create a character that responds
to voice-over-IP data. The overall effect of this is a simulated "video-phone",
where a proxy responds to the user's voice in a way chosen by the user.
In all, FaceScript™ represents a complete solution for implementing facial
animation. By incorporating deep knowledge of the motion and structure
of the human face, FaceScript™ gives maximum leverage to designers, animators
and end-users, allowing full control with a minimum of specification.