The full guide is available for Premium Members. If you are a non-member, please enjoy the Quick Table and Comprehensive Guide sample for “eyebrow actions.”
difficulties with AR development toolkits
If you or your team are using open-source face tracking kits to:
animate faces
overlay virtual content
create expression-based events
. . . figuring out what’s what can be challenging – especially if you or your team do not have a strong background in:
facial expressions
face tracking
the Facial Action Coding System (FACS)
vaguely-defined items
Face tracking software development kits (SDKs) can be confusing for a number of reasons. A large contributor to this confusion is the lack of detailed documentation available for users.
Human facial expressions are complex and rich with nuance. Understanding how to identify and break down expressions is not always simple or intuitive. Despite this reality, face tracking kits often only provide minimalist definitions for the expression shapes in their libraries.
Minimally-defined expression shapes:
create room for user misinterpretation.
increase user’s likelihood to confuse similar-looking shapes.
limit the user’s potential to effectively use the product.
getting around the ambiguity
Expression shapes in most face tracking products (despite their names) are primarily FACS-based. For those who may be skeptical: FACS is derived from anatomy; so unless a face kit has completely annihilated the foundations of human facial anatomy, all shapes will have FACS equivalents. If you wish to foster a better understanding of the face tracking products you are using, you should familiarize yourself with FACS.
FACS naming is standardized. FACS is consistent. Each FACS shape has a detailed, well-defined, and heavily researched description. If you are well-versed in FACS, you can equip yourself with the tools you need to compensate for the ambiguity of most expression libraries.
Whether or not you are FACS-savvy, if you want a clearer breakdown of ARKit facial expression shapes, this is the document for you 🙂
relaxation of levator palpebrae superioris & contraction of the palpebral portion of orbicularis oculi
eyeLookUp (left & right)
M63 – eyes up
superior rectus & inferior oblique**
eyeLookDown (left & right)
M64 – eyes down
inferior rectus & superior oblique**
eyeLookIn (left & right)
AU66 – crosseye (when applied to both eyes at once); otherwise, eyeLookInLeft must be paired with eyeLookOutRight for rightward gaze (FACS M62), while eyeLookInRight must be paired with eyeLookOutLeft for leftward gaze (FACS M61)
medial rectus**
eyeLookOut (left & right)
AU65 – walleye (when applied to both eyes at once); see above for leftward vs. rightward gaze
(AU26) pterygoids, digastric muscles or (AU27) masseter, relaxed temporalis and internal pterygoid relaxed muscles
jaw (left & right)
AD30 – jaw sideways
pterygoids and temporalis
jawForward
AD29 – jaw thrust
pterygoids and masseter
** = Indicates an oversimplification of muscular correlates. For example, the muscles related to gaze are much more complex than “superior rectus = eyes look up.” Rather, they work together in a system of relativity. For the purposes of this guide, however, movements are matched with the most relevant and idealistic muscles.
COMPREHENSIVE GUIDE sample
eyebrow actions
all
browInnerUp
browOuterUp (left & right)
browDown (left & right)
all
browInnerUp
browOuterUpLeft & browOuterUpRight
browDownLeft & browDownRight
browInnerUp
ARKit term & description
browInnerUp
The coefficient describing upward movement of the inner portion of both eyebrows.
________________
FACS-friendly term and description
AU1 – inner brow raiser
The upward movement of the inner portion of the eyebrows.
Caused by the contraction of the inner portion of the frontalis muscle.
Due to frontalis muscle variability, AU1’s effects on the eyebrows (e.g. how close the upward pull is to the inner brow tip) may differ across individuals. (See Frontalis Variation.)
May only be useful to separate for complex shape creation.
browOuterUp (left & right)
ARKit term & description
browOuterUpLeft The coefficient describing upward movement of the outer portion of the left eyebrow.
browOuterUpLeft The coefficient describing upward movement of the outer portion of the right eyebrow.
________________
FACS-friendly term and description
AU2 – outer brow raiser
The upward movement of the outer portion of the eyebrows. May move somewhat diagonally in certain people.
Caused by the contraction of the lateral portion of the frontalis muscle.
Due to frontalis muscle variability, AU2’s effects on the eyebrows (e.g. where the main point of focus for the outer pull is in relation to the inner pull) may differ across individuals. (See Frontalis Variation.)