“it’s all in the eyes” and other lies: a critique on contemporary emotion research

felt and unfelt smile poll - authentic - genuine - inauthentic

While Ekman’s research and writings have inspired many of us (myself included) – tech, academia, and entertainment have become too reliant on the work of this single contributor and his followers. The overwhelming majority of companies I’ve worked for or contracted with have either been wholly blind to and/or disinterested in alternative branches of thought. Being that I have worked for Big Tech and influential emotion tracking startups: This is unsettling.

With the progression of tracking technology and the increasing demand for realism in digital art, it’s beyond time that these industries (and likely many others) adopted a more comprehensive approach to understanding emotions and facial expressions.


facial actions in context

upper lip raiser and nasolabial furrow deepener

Because of its name, upper lip raiser is considered the go-to action for lifting the top lip and tends to get overused in art and tech, acting as a replacement for nasolabial furrow deepener or a shortcut for showing teeth during smiling.

Despite its name, upper lip raiser is not the only lip-raising action! In fact, applying upper lip raiser to contexts better suited for nasolabial furrow deepener can be detrimental to the essence of a target expression.


faces you don’t want to see during UX research – especially for VR

facial expressions you to avoid during UX research sessions

Faces of discomfort often followed headset adjustment – or predicted upcoming adjustments. Bored faces and faces on the contempt spectrum tended to be predictive of undesirable experiences later disclosed during the post-demo interviews.

These expressions were not just useful for predicting events. They also served as points for further investigation. If a user made a particular face on multiple occasions when discussing or experiencing a certain event, we were able to press further and gather deeper insights.


faking aging in characters

The Falcon and the Winter Soldier - Carl Lumbly actor

If you are aging a face, pay attention to where you add sagging, deep lines, and folds. There are patterns to follow. While everyone’s pattern is different, general principles still exist. Aging reflects many things – our unique anatomy, our repeated expression use, our past injuries, etc. It is a map of our history.


bias in emotion tracking

AU23 - lip tightener - Facial Action Coding System - FACS

We seem to subscribe to the popular oversimplification that machines are less biased than humans; however, if you are familiar with the ways in which machines are trained to read and focus on different aspects of data, you will know: It’s just not that simple.

Machines are not free of bias if they are trained by humans.