Lip Sync

The KEY to lip sync is getting the FEELING of the WORD and NOT the INDIVIDUAL LETTERS
— Richard Williams

Don’t animate every syllable

When animating lip sync your first instinct might be to try to pose out the mouth shape for every syllable that you hear in the audio. However the idea is not to over work the poses, get the shape of the word and make sure the viewer can read it . Select and focus on the most important mouth shapes in the audio and blend over the rest, this is referred to as "phrasing". This is how humans talk. When we speak we don't articulate every syllable and letter. Listening to each word and the syllable combinations, you can usually break them down into a variation of ten phoneme sets. Whats a phoneme;

"Within linguistics there are different views as to exactly what phonemes are and how a given language should be analysed in phonemic terms. However, a phoneme is generally regarded as an abstraction of  a set of speech sounds which are perceived as equivalent to each other in a given language."  Wikipedia

 
 

 

  • A and I: For the A and I vowel sounds, the lips are generally pulled a bit wider, teeth open, tongue visible and flat against the floor of the mouth.
  • E: The E phoneme is similar to the A and I, but the lips are stretched a bit wider, the corners uplifted more, and the mouth and teeth closed a bit more.
  • U: For the U sound, the lips are pursed outwards, drawn into a pucker but still somewhat open; the teeth open, and the tongue somewhat lifted.
  • O: Again the mouth is drawn to a pucker, but the lips don't purse outwards, and the mouth is rounder, the tongue flat against the floor of the mouth.
  • C, D, G, K, N, R, S, Th, Y, and Z: Long list, wasn't it? This configuration pretty much covers all the major hard consonants: lips mostly closed, stretched wide, teeth closed or nearly closed.
  • F and V: Mouth at about standard width, but teeth pressed down into the lower lip. At times there can be variations closer to the D/Th configuration.
  • L: The mouth is open and stretched apart much like the A/I configuration, but
  • M, B, and P: These sounds are made with the lips pressed together; it's the duration that matters. "M" is a long hold, "mmm"; "B" is a shorter hold then part, almost a "buh" sound; P is a quick hold, puff of air.
  • W and Q: These two sounds purse the mouth the most, almost closing it over the teeth, with just the bottoms of the upper teeth visible, sometimes not even that. Think of a "rosebud mouth".
  • Rest Position: Think of this as the "slack" position, when the mouth is at rest--only with the thread of drool distinctly absent.

No two people express themselves in an identical fashion, and each has individual facial quirks that make their speech and expressions asymmetrical, so play with your characters poses to to try and match their emotion or character traits.

 
 

Offset the lip sync for readability

Offsetting lip sync is something that is often forgotten or overlooked, but it is important to take into account. With animation, its all about readability and lip sync is no exception. For your audience to be able to read the lip sync you’ll want to offset the jaw opening one to two frames before the audio is actually heard. If you have the jaw opening and closing each frame that the audio is heard your lip sync will feel like it’s just slightly ahead of the audio.

Hit those closed mouth shapes

When it comes to mouth shapes some of the most important ones to hit properly are when the mouth is closed, like B, M, and P. You can’t blend over these because you need them to read clearly. To do that you’ll want to hold them for a couple frames, and pop open to the next mouth shape. For example, say your character is saying the word “but”. If you blend over the “B” shape it will look like the mouth is saying “ut.” To get the word to read properly, you’ll want to hold the “B” shape for 2 frames and then pop open to “ut.” Try it out and you’ll see the difference!

Create a Dialogue Chart

As we mentioned earlier, planning out the audio is incredibly important when working on a shot with dialogue. Write down each word of dialogue and then break it up into syllables. Track the timing for each word and study the high points, or the areas where the character speaks higher in the audio. Most words should have one of these high points in them. Take notes on what syllable it’s at and use them as guides on where to push the lip sync in your animation.

 
 

Act out the scene

Animators are actors. The best way to get ideas for the scene is to get in front of a camera and act out the shot. Don’t be shy, all animators do it. Try to become the character during the acting process. Play the audio in the background and act out the scene over and over. When you think you’ve got the shot, playback the video you’ve captured to double-check. Only stop when you’re entirely happy with your performance.

Block in the key poses

Try and create proper story telling poses that describe the facial animation. For each story telling pose, you should make sure to pose each area of the face and not just the entire head. This will help you see the type of facial pose that you want to hit and at what word in the dialogue. Try to keep the poses to a minimum; you just want to get the most important ones in at this point.

Exaggerate the movements

Make sure the timing is exactly how you planned it and your holds are in the right spot. Once you’re happy with the timing, go in and start pushing some of the poses to really exaggerate the facial movement. For example, when the character brings his head down, drag the eye brows and have them offset a few frames. You could even utilize the scale attributes for the head to add some squash and stretch. Exaggerating movement is just as important with facial animation as it is in the rest of the body.

Eye Brow Animation

It’s important to have strong eye brow poses to sell the facial animation. Don’t overdo it by having the eye brows moving during every single word. Instead, find a few important eye brow shapes that you want to hit during the dialogue. In the ogre animation there are really only two main eye brow poses: Frowned, and surprised. It’s how you transition between the two that helps sell the animation.

Add in Blinks

When adding blinks to your facial animation, make sure that you aren’t adding them just for the sake of it. To add blinks with the correct timing you need to think about the emotional state of the character. Is he angry, happy, sad or shocked.