ARKit to FACS Cheat Sheet

The full guide is available for Premium Members. If you are a non-member, please enjoy the Quick Table and Comprehensive Guide sample for “eyebrow actions.”

difficulties with AR development toolkits

If you or your team are using open-source face tracking kits to:

- animate faces
- overlay virtual content
- create expression-based events

. . . figuring out what’s what can be challenging – especially if you or your team do not have a strong background in:

- facial expressions
- face tracking
- the Facial Action Coding System (FACS)

vaguely-defined items

Face tracking software development kits (SDKs) can be confusing for a number of reasons. A large contributor to this confusion is the lack of detailed documentation available for users.

Human facial expressions are complex and rich with nuance. Understanding how to identify and break down expressions is not always simple or intuitive. Despite this reality, face tracking kits often only provide minimalist definitions for the expression shapes in their libraries.

Minimally-defined expression shapes:

- create room for user misinterpretation.
- increase user’s likelihood to confuse similar-looking shapes.
- limit the user’s potential to effectively use the product.

getting around the ambiguity

Expression shapes in most face tracking products (despite their names) are primarily FACS-based. For those who may be skeptical: FACS is derived from anatomy; so unless a face kit has completely annihilated the foundations of human facial anatomy, all shapes will have FACS equivalents. If you wish to foster a better understanding of the face tracking products you are using, you should familiarize yourself with FACS.

FACS naming is standardized. FACS is consistent. Each FACS shape has a detailed, well-defined, and heavily researched description. If you are well-versed in FACS, you can equip yourself with the tools you need to compensate for the ambiguity of most expression libraries.

Whether or not you are FACS-savvy, if you want a clearer breakdown of ARKit facial expression shapes, this is the document for you 🙂

ARKit to FACS Translation Guide

QUICK TABLE

ARKit label

corresponding FACS name(s)

corresponding muscle(s)

browInnerUp

AU 1 – inner brow raiser

frontalis, medial portion

browOuter (left & right)

AU2 – outer brow raiser

frontalis, lateral portion

browDown (left & right)

AU4 – brow lowerer

corrrugator supercilii, depressor supericilii, procerus

eyeWide (left & right)

AU5 – upper lid raiser

levator palpebrae superioris

cheekSquint (left & right)

AU6 – cheek raiser

orbicularis oculi, orbital portion

eyeSquint (left & right)

AU7 – lid tightener

orbicularis oculi, palpebral portion

eyeBlink (left & right)

AU45 – blink

relaxation of levator palpebrae superioris & contraction of the palpebral portion of orbicularis oculi

eyeLookUp (left & right)

M63 – eyes up

superior rectus & inferior oblique**

eyeLookDown (left & right)

M64 – eyes down

inferior rectus & superior oblique**

eyeLookIn (left & right)

AU66 – crosseye (when applied to both eyes at once); otherwise, eyeLookInLeft must be paired with eyeLookOutRight for rightward gaze (FACS M62), while eyeLookInRight must be paired with eyeLookOutLeft for leftward gaze (FACS M61)

medial rectus**

eyeLookOut (left & right)

AU65 – walleye (when applied to both eyes at once); see above for leftward vs. rightward gaze

lateral rectus**

noseSneer (left & right)

AU9 – nose wrinkler

levator labii superioris alaeque nasi

mouthUpper (left & right)

AU10 – upper lip raiser

levator labii superioris

mouthSmile (left & right)

AU12 – lip corner puller

zygomaticus major

mouth (left & right)

See notes in Comprehensive Guide.

mouthDimple (left & right)

AU14 – dimpler

buccinator

mouthFrown (left & right)

AU15 – lip corner depressor

depressor aguli oris

mouthLowerDown (left & right)

AU16 – lower lip depressor

depressor labii inferioris

mouthShrugUpper

See notes in Comprehensive Guide.

mouthShrugLower

AU17 – chin raiser** mostly
See notes in Comprehensive Guide.

See notes in Comprehensive Guide.

cheekPuff

AD34 – puff

buccinator

mouthPucker

AU18 – lip pucker

incisivus labii superioris, incsivus labii inferioris

mouthStretch (left & right)

AU20 – lip stretcher

risorius

mouthFunnel

AU22 – lip funneler

orbicularis oris

mouthPress (left & right)

AU24 – lip presser

orbicularis oris

mouthClose

See notes in Comprehensive Guide.

cheekPuff

AD34 – puff

buccinator

mouthRoll (upper & lower)

AU28 – lips suck

orbicularis oris

jawOpen

AU26 – jaw drop or AU27 – mouth stretch

(AU26) pterygoids, digastric muscles or (AU27) masseter, relaxed temporalis and internal pterygoid relaxed muscles

jaw (left & right)

AD30 – jaw sideways

pterygoids and temporalis

jawForward

AD29 – jaw thrust

pterygoids and masseter

** = Indicates an oversimplification of muscular correlates. For example, the muscles related to gaze are much more complex than “superior rectus = eyes look up.” Rather, they work together in a system of relativity. For the purposes of this guide, however, movements are matched with the most relevant and idealistic muscles.

COMPREHENSIVE GUIDE sample

eyebrow actions

all

browInnerUp

browOuterUp (left & right)

browDown (left & right)

all

browInnerUp

browOuterUp (left & right)

browDown (left & right)