How to increase posed expression quality:
Posed expression capture systems are becoming more and more popular. Though they have been around in the art/entertainment industry for quite some time, in recent years, they have also gained traction in the tech industry. Regardless of industry, there are universal challenges in acquiring desired poses. To help achieve higher quality data for machine learning in tech and to promote more accurate reference poses in art/entertainment, below is a guide with common mistakes to avoid in order to get the most out of your data participant, actor, or model.
NOTE: There is a lot to cover on this topic. This is a truncated version with only upper face expressions. I’m available for consultation (facetheFACS@melindaozel.com) if you would like the same information for the lower face and/or combination expressions.
Best Practices for Capturing Posed Expressions in Technology
Mistakes to Avoid in Hiring and Planning
Choosing your collection team will greatly impact the quality of your project. In tech, common mistakes made when hiring data acquisitionists include:
🚫 DON’T hire over-qualified individuals to collect the data 🚫
- Data capture can be tedious and repetitive. While expertise is important in the data strategy phase, it is not necessary in the data acquisition phase. Hiring over-qualified individuals to collect data will increase the likelihood of your employees growing tired from under-stimulating tasks and career stagnancy. Boredom and stagnancy will reduce incentive for individuals to remain on your team – unless you have a clear path for advancement. Hiring individuals with less experience will increase chances of retaining members of your team. It will also provide career growth opportunities for inexperienced employees rather than perpetuating stifling contractor-based structures.
- One of the worst things you can have in a data capture project is a high turnover rate. You want your team to know the ins and outs of your house-style collection process. A well-informed team that understands the struggles and hacks of your system will yield more accurate data and reduce time spent troubleshooting.
🚫 DON’T value experience over people skills 🚫
- Getting your participant to feel comfortable is half the battle. People are less expressive in novel and unnatural settings. Data capture environments are inherently unnatural; so you will want to make up for the uncomfortable environment with an intuitive and personable staff. Hiring data acquisitionists with backgrounds in customer service or with strong emotional intelligence will take you a lot further than hiring individuals with technical experience.
The expression data you choose to capture will vary depending on use case and resource availability. Common mistakes in planning include:
🚫 avoid deficient guides and inaccurate visuals 🚫
- Due to lack of knowledge in expression nuances, most posing guides have either incorrect descriptions or inaccurate visual examples. To reduce this problem, it is crucial to consult with experts or dedicate time/training materials (like the Facial Action Coding System Manual) to designated members of your team.
🚫 avoid redundant or irrelevant poses 🚫
- A common issue with expression capture is redundancy. While it is important to gather a diverse expression set with various poses/pose combinations, it is crucial to not waste energy on unnecessary poses. Some facial actions combine intuitively; some do not. Some upper face expressions impact the look of the lower face, and some lower face expressions impact the look of the upper face. Some do not. If your team is mindful of these impacts, you will be able to reduce nonessential pose combinations.
- For example, lip corner depressor is a lower face action that has little to no impact on the upper face. There is no immediate need to combine lip corner depressor with upper face shapes – unless you need to capture a full sadness expression.
🚫 avoid leaving out important shape combinations 🚫
As mentioned above, some shapes combine intuitively and some don’t. Being aware of how facial actions combine will allow you to create a diverse but streamlined set of expression poses. Which poses you need will depend on your available resources and underlying product needs.
🚫 avoid unstandardized nomenclature 🚫
- Standardized terminology for each base expression already exists. Using standardized terminology will reduce miscommunication. Do not name your own poses unless you need poses that do not already have standardized names.
- An example of miscommunication caused by ambiguity of unstandardized terminology is the the word “frown.” Depending on your country of origin, the term “frown” can refer to either brow lowerer or lip corner depressor. In North America, “frown” refers to lip corner depressor – hence the phrase, “Turn your frown upside down.” Such a saying would make no sense outside North America, as the rest of the world uses “frown” to indicate brow lowerer.
- The best current naming system is the Facial Action Coding System. FACS is widely used and features clearly defined terms with detailed requirements.
- If you require poses that do not have official names, it is crucial to document your working definitions.
🚫 avoid failing to encourage cross-functional communication 🚫
Data acquisition teams, data annotation teams, engineering, research, and product management must stay in open communication. Data needs and quality are affected at all levels of productization. It is imperative to foster cross-functional communication between all roles to highlight different perspectives and insights.
Best Practices for Capturing Posed Expressions in Both Technology and Art/Entertainment
Mistakes to Avoid During Capture
🚫 DON’T disclose difficulty 🚫
- Many poses are challenging for people to hit. Do not tell your participant or actor when a pose is difficult until they have already attempted and failed. Each individual will struggle with different poses. Preemptively disclosing difficulty will bias your participant and/or cause them to overthink something that may have otherwise come naturally to them.
🚫 DON’t make participant sustain poses 🚫
- Certain facial muscles are not designed to be held for long periods of time. Some data processes require participants to hold poses for as long as three to five seconds. Such long duration will cause your model to experience expression fatigue. Expression fatigue greatly increases likelihood of co-activating unwanted facial muscles.
🚫 DON’T force participant to pose expressions they cannot 🚫
- As mentioned above, challenges in hitting poses will differ across individuals. Some individuals will never be able to achieve certain poses. This inability may be the result of an individual’s lack of muscle control, but it can also be a product of anatomical diversity.
- Anatomical diversity in facial muscles is an under-discussed and under-researched topic. Various cadaver studies have identified significant morphological differences in our facial muscle makeup. In some studies, for example, as much as 40% of cadavers examined were missing the risorius muscle. Risorius is the muscle behind lip stretcher, a common target for basic pose shapes as well as emotion prototypes. Failing to consider morphological diversity will not only waste your time and your participant’s time – but it will cause unnecessary stress for the participant and acquisitionist.
Challenges in Expression Posing in Both Technology and Art/Entertainment
AU 1 – inner brow raiser
- Only a small percentage of individuals can activate inner brow raiser without employing brow lowerer or outer brow raiser. If you have the luxury of time to train your participant to isolate their brow movements – great! However, it may be best to move on if they cannot hit it in the first few tries.
AU 6 – cheek raiser
- Cheek raiser is often contaminated with lid tightener and vise vera. However, one hack to get participants to at least locate their cheek raiser is to ask them to wink. Have them alternate winking on each side of their face while asking them to feel the movement in their cheeks. Though this trick should help create the feel for the muscle, it will not guarantee omission of lid tightener.
- Cheek raiser is also mistakenly replaced by or combined with lip corner puller (mouth “smile”). While it may not be possible to separate cheek raiser from lid tightener in most participants, it is imperative to spend extra time to ensure participant is not co-activating lip corner puller. Lip corner puller affects the shape of the cheeks too much and will drown out the features from cheek raiser.
AU 7 – lid tightener
- As mentioned above, people often co-activate lid tightener with cheek raiser. As you increase the strength of lid tightener, the probability that your participant will activate cheek raiser increases dramatically. One way to mitigate this co-activation is to opt for a lower intensity lid tightener pose.
AU 9 – nose wrinkler**
**While nose wrinkler is technically considered a lower face action, it significantly impacts all areas of the face, including the eyes and eyebrows.
- The most common issues associated with nose wrinkler will appear in different parts of the face. Sometimes an individual can completely fail to employ nose wrinkler and instead activate brow lowerer or upper lip raiser.
- Getting rid of this problem can be as simple as asking your participant to lightly touch the bridge of their nose. This trick helps bring attention to the target area.
- If the nose bridge trick does not work, have your data acquisitionist give more thorough instructions to differentiate brow lowerer and upper lip raiser from nose wrinkler.
✨✨ The lower face is much more complicated than the upper face. Please contact me (facetheFACS@melindaozel.com) for consultation on how to get the best out of posed lower face expressions. ✨ ✨