Lip Syncing in Flash

Hi,

Lip syncing is a crucial part of making a good flash animation, or any movie really, unless your making some emotionally deep silent film, or high action fighting movie like killbunny. Its pretty important to have dialogue in a movie, and its equally good to have good mouth movements for pronouncing those words. Dialogue and lip syncing is what makes or breaks a video.

In case your not familiar with the term, lip syncing is animating some kind of mouth to accurately sound words or speech, to give the illusion of a speaking character.

Anyway, ive noticed recently a lack of decent lip syncing tutorials and a large amount of bad short silent animations. bad in that they are animated with above average skills, to make pointless confusing soundless animations that make no sense, like a stick figure waving at a balloon then getting decapitated by an alien space ship.

I consider that a waste of talent.

Recently while working on my Pokemon parody, i have discovered the secrets of lip syncing, and that there are many effective approaches. Although being the stubborn lazy bum that i am i didn't bother to apply these, and i kept to my original method of frame by frame sounding out the sounds, which in my opinion looks better. I discovered there is a surprisingly little known method of playing sound in flash that makes lip syncing super easy, next i found out there is an extension for flash CS5 that automatically lip syncs in the same way, though in my opinion it can never replace hand animated speech, not on its own anyway.

Still, i am talking about lip syncing methods used in tv shows such as stoked and total drama island, which recently have even gone fully automatic. Lip syncing now is more about being able to draw a good mouth sounding a syllable than actually doing fluent talking.


I will show several methods for lip syncing all of which can produce good to amazing results regardless of your skills,
all of these methods though incorporate a surprisingly little known method of altering the way flash handles sound on the main timeline, the way it works is very strange, and seems almost as though its meant to be used just for lip syncing on the timeline.

Step 1: Changing How Flash Plays Sound on the Timeline

This mysterious method i discovered was so simple and so amazingly helpful i cant beleive its barely ever mentioned in tutorials.

This method is called streaming, in flash it is a syncing option, locking sound to the timeline and cutting up your audio into chunks which are played on each and every frame.

To access it, you need to place an audio file onto a frame, then select that frame, in the timeline , while the properties tab is open.
In the properties tab, your sound options will appear, including what file is to be played on that frame, any effects like fading, and finally, the syncing, how the sound will play in respect to the timeline. By default, all sounds play as event, as this allows for the best quality.
anyway, to bind your sound to the timeline, change the option from event to streaming.
Also take note of the start and stop, using those will mean you dont need to cut music short before importing if you only want to play to a certain point

This means in flash, when you are animating and you drag the frame marker up or down the timeline, as it goes over that frame it will play the chunk of the sound bound to that particular frame.
This beats the alternative of animating a few frames at a time based on what sound you asume might be played on that frame, by playing just before the sound, letting it play out , since it wont stop playing in "event" mode until the audio is done, then drawing what sound you think is being said at that time.

Incidentally, if you export your movie with any of the sounds syncing methods as "stream", then it will mess up the syncing of all sounds in the movie, also, the chunks bound to each frame can only hold a limited amount of bytes, so the bitrate will drop dramatically and your sound will sound like it was recorded on a phone or really bad cheap microphone.

once you finish animating you can change the sound syncing option back to event which will play the sound as normal in the swf, only in its actual quality.

Step 2: Key Frame Lip Syncing

The first is one of the simplest methods, and is the base for several others. it provides one of the easiest ways to get a fairly authentic mouth movement with the least effort, other than the binary method which is just drawing a circle for whether the mouth is opened or closed, which i wont go into.
This method is key frame sounding. What this means is that you draw your mouth movements at the points where there is the most change, which would be when pronouncing a different sound. e.g silent to "GA" or "TA" to "MOO".
Once you master this method, the others will be much easier, which essentially just go further into this one.

Once you have your audio set to streaming, and your characters face all ready to go, drag the frame marker over the frames and listen for important syllables that would have the most impact on how the mouth is shaped.  e.g AH EE OO EH OH and AY . If you ever learnt Japanese its essentially the Japanese vowels in most cases

Now draw on those frames the mouth for that sound. In several methods this is how you start . My example is the hobo from my parody saying i am a hobo, embedded are 3 versions for the steps you need to take.

As i said, first is to just draw the biggest changes in sound as well as what would create the most change in mouth shape.


Next is to right before and right after each of these mouthed sounds, to put in a transition, what the mouth should look like right before or right after that sound. Its best to keep a frame or 2 between key frames if there is a lot of distance between the sounds you've sounded . in many cases the sound right before or after will be something like te , be ,pe, ect. dont sound those with a closed mouth.


Lastly, wherever applicable, draw a closed mouth. Don't use it too much though or else it sort of ruins things. In this example i have changed the syncing option from stream to event, so you can see what kind of difference it makes in audio quality

If i had any other sounds on the timeline, streaming or event, they would all play at the wrong times under most circumstances, so when exporting be sure to set to event.

Step 3: Key Frame Tweened Lip Syncing

This method is based off the first and is used most commonly in the flash based tv shows you see around so much.
This method is often used in unison with an automatically lip synced animation, in which the first step we did just then, is done automatically.

This method involves using motion or shape tweens to animate the mouth during clear keyframes. for those of you unfamiliar with the types of keyframes, there are 3, keyframes, blank keyframes (empty keyframes) and clear keyframes (frames that just display whatever the last keyframe contained.

During the clear keyframes, no movement happens , while in reality, you or your voice actors mouth is likely moving, thus yours should also. What we do is, establish how exactly the mouth will be moving between the current keyframe and the next keyframe assuming there are 2 or more clear keyframes between them. Tweening involves using moving or warping an object to another location or size, and flash will automatically do a frame by frame transition.
If you have 1 or 2 clear keyframes distance is often better to just hand animate or manually transform the mouth.

An important thing to remember is you can not to classic motion tweens or shape tweens on a layer that has motion tweens on it, they interfere and the symbols often get changed around. to my knowledge. also remember, motion tweens have a straight line velocity, the speed is always constant. where applicable, use the motion editor to slow down or speed up part of the tween to match the sound.

This method doesn't require any further drawing, often you'll find people have pre drawn mouths which they place instead of drawing, then they tween them to give the illusion of further animation when they are just being warped. Personally i don't like it because as an animator, to me it shows laziness, especially when it could easily look better with frame by frame.

An important thing to remember when lip syncing and tweening is to keep the level of the upper lip constant. i havent done such a good job of that here, as well as the lines getting thinner and thicker, so its very noticeable.

Step 4: Frame by Frame Hand Animating

This method is my preferential favorite as i always wind up getting it to look more or less as i want it to.

Essentially this method is just filling the gaps like the tweening method, except, you draw it by hand, wherever you can.

This method is preferential to tweening whenever the characters head isn't in motion or when you are using pre-drawn mouths that would take too long to alter/ redraw correctly.

The only trick is to always make sure you are drawing the right shape for the mouth and at the right time, namely not closing the mouth off to a line because of silence during the middle of a word.

The example of frame by frame is a few clips from my pokemon parody.


The first scene is an example of full frame by frame animation, hardly used professionally, but still as effective. Essentially i hand drew every single frame used in the speech with no gaps or pauses.
The next scenes however, i use something more along the lines of key frame syncing. The hobo when saying "change", is shape tweening, as that is an appropriate time to use a tween. Being a shape tween, it maintains stroke width while morphing.

As you can see, full frame by frame when done right, is very smooth and effective, however if you look carefully at it, pacing it right, youll find it doesnt really match up exactly how it should be expressed, but unless you look really hard you dont see it. This is what i mean, whether its a bit off or right on the dot, it always works. The more breaks and pauses used however, any errors become much more apparent.

Step 5: Choose the Style You Feel Most Comfortable With

Start off with whatever style you feel most comfortable doing and work from there. Over time however, its good to try using key frame animation, tweens and transformations  and Frame by frame animation wherever tweens wont work, all in the same places, or look at how your favorite flash based tv shows do it, often youll find frame by frame is used as little as possible as professionals preffer to stop their animations being "shaky".

Most of the time on TV they get around their cheap methods by animating on a high framerate (30-50fps) in order to get all their sounds exactly at the right instances, as well as having the character being very emotive with body language.


For those who havent a clue about any speech related animation that are lucky enough to be starting with access to flash CS5, there is an extension called SmartMouth, which automatically lip syncs based on the "phoneme" of that frame.

Heres a link : http://www.adobe.com/devnet/flash/articles/lip-sync-smartmouth.html
As you can see though, by the example shown on the site, its not always best to rely solely on what the extension makes for your lip syncing, some touching up here and there and manual animation is required before its worthy of a thumbs up.


This concludes my guide.
If you have any questions please comment and ill respond as soon as possible. Also please take the time to rate this instructable with the rank you think it deserves.
Also if your stuck or having difficulties animating or lip syncing in flash i would be glad to help out, just leave a PM.


Lastly, for anyone interested in my pokemon parody, the digital backup (updated as i make it), can be found here http://www.jpoopdog.com/swf/pokemon_treefail.swf


Attached is the final copy of the Example FLA. Everything was created in adobe Flash CS4.