ARKit to FACS Translation

difficulties in AR development toolkits

(Scroll down to skip to the guide.)

If you or your team are using open-source face tracking kits to:

    • animate faces
    • overlay virtual content
    • create expression-based events

. . . figuring out what’s what can be confusing – especially if you or your team do not have a strong background in:

    • facial expressions
    • face tracking
    • the Facial Action Coding System (FACS)

vaguely-defined items

Face tracking software development kits (SDKs) can be confusing for a number of reasons. A large contributor to this confusion is the lack of detailed documentation available for users.

Human facial expressions are complex and rich with nuance. Understanding how to identify and break down expressions is not simple or intuitive. Despite this reality, face tracking kits often only provide vague definitions for the expression shapes in their libraries.

Poorly-defined expression shapes:

    • create room for user misinterpretation
    • increase user’s likelihood to confuse similar-looking shapes
    • limit the user’s potential to effectively use the product

the frown example

When I was at Facebook / Oculus, I worked with many revered research scientists in expression tracking. Even these top minds had trouble describing, identifying and differentiating facial expressions – subtle or not. My coworkers’ inability to clearly communicate the expressions they were referring to led to frequent misunderstandings and miscommunications.

One specific example of miscommunication arose whenever anyone on my team used the word “frown” to describe a facial expression of sadness or disappointment. Depending on where you are from, “frown” can have different meanings.

“Although most technical definitions define it as a wrinkling of the brow, in North America it is primarily thought of as an expression of the mouth . . . The mouth expression is also commonly referred to in the colloquial English phrase ‘turn that frown upside down’ which indicates changing from sad to happy.”
—- Wikipedia

This linguistic debacle is just one example of the many potential misinterpretations that can transpire when discussing facial expressions.

getting around the ambiguity

Expression shapes in most face tracking products (despite their names) are primarily FACS-based. If you wish to foster a better understanding of the face tracking products you are using, you should familiarize yourself with FACS.

FACS naming is standardized. FACS is consistent. Each FACS shape has a detailed, well-defined, and heavily researched description. If you are well-versed in FACS, you can equip yourself with the tools you need to compensate for the ambiguity of most expression libraries.


Whether or not you are FACS-savvy, if you want a clearer breakdown of ARKit facial expression shapes, here ya go:

(NOTE: I am starting with the brows & eyes and will add more to this next Monday – August 10, 2020.)

ARKit Translation Guide

the brows

the eyes

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.