Streamlining Opacity Keying

[jgermeten]

Working on a lip syncing job, switching pre-made phoneme images to match recorded audio.

I've been keying opacity manually using the sliders, but that takes a lot of time to go back and forth with my eyes, making sure I'm on the right layer, avoiding mis-clicks, making sure I'm actually getting the opacity to zero, etc. 

Is there a single-key shortcut for keying a layer's attributes like opacity? The fastest I can get is copy-pasting 0% and 100% keyframes, which is a little better than the slider, but still it would be nice to have a single click-on-the-layer-hit-the-key to adjust opacity. Is this a thing already and I just don't know it? Or are there scripts and plugins to help with this?

Any help is appreciated. Thank you!

[Rick Gerard · Accepted solution]

The most common way to do phoneme animation is to create a short image sequence with one mouth position for each frame, apply time remapping, then just enter the frame number of the frame you want. If you make up a chart and set the time display to frames it is very easy. 

a = 1

o = 2 

and so on. 

You typically set all keyframes to hold keyframes. Everything on one layer and one number entered for each sound.

Here is a fairly decent tutorial on the process. He leaves out a few shortcuts like using the j and k key to jump between markers, and I would use a slightly different approach with the expression and the slider, but it should get you started.

If you really want to go crazy you can number the markers on the audio track and have an expression use the marker number as the frame number. This will eliminate all of the manual keyframing so you only have to do the work of syncing numbers to the audio once. 

If you do not have access to Audition you can add markers to the Audio track and name them in After Effects. I do it all the time. I use markers almost every time I edit anything to audio. I use this expression to turn layer marker names (comments) to frame numbers.

mrkrName = thisComp.layer(index + 1).marker.nearestKey(time).comment;
framesToTime(mrkrName, fps = 1.0 / thisComp.frameDuration)

Put the audio track with markers just below the Phoneme animation layer and all you have to do is add the markers and number them by double-clicking. To fine-tune the timing all you have do is slide the markers. 

I have a similar expression that I use to add subtitles. In the last line, framesToTime..., is simply removed to generate the text. A little additional code is added to an opacity text animator to automatically face up the words and fade them out using the description in the layer markers and the driving force for all of the animations. 

[jgermeten]

Thank you so much! This is much better than what I was looking for.

If I can ask a followup, more about understanding the script,

mrkrName = thisComp.layer(index + 1).marker.nearestKey(time).comment;
framesToTime(mrkrName, fps = 1.0 / thisComp.frameDuration)

mrkrName opens the definition of a new variable

thisComp refers to the present composition

.layer() opens our capacity to define which layer's markers are being referenced

"index" is the layer with the script

+ x changes the referred layer to x positions below "index"

.marker tells the program that we're looking for marker attributes on that layer

.nearestKey seems to be about finding the nearest key to the marker? Like in case you've used Audition and the markers aren't quite lined up with the frames?

.comment is fetching the value of the marker's comment

framesToTime() (I'm guessing?) is AE's in-house call to use certain frames as reference for how long to show a certain frame?

mrkrName takes the variable we've defined above, namely taking the value within the audio track's comment

fps = 1.0 (again, guessing) simply creates a numerator which can be altered to adjust the duration, which is determined in part by

thisComp.frameDuration, which in this particular case is 1 since each phoneme lasts a single frame within its own subcomposition. If you wanted it to last half as long, for example, you would write

(mrkrName, fps = 2.0 / thisComp.frameDuration)

right?

Also I've noticed that for some reason the phoneme is changing a bit before the actual marker, and it's skipping some markers when they're a single frame between.. It's after the halfway point between the two markers, so my guess is that it's rounding up from x.5, but do you have any idea why that's happening? Do you know how to constrain it to whole numbers?

Sorry, I'm kind of new to this sort of thing..

Again, thanks so much for your help!

[Rick Gerard]

You have everything figured out except the fps.

Time is always measured in seconds. When you add a keyframe value to time remapping it is always in seconds. To convert seconds to frames you divide 1 by the frame duration. Changing the 1 to a 2 throws off the calculation so the wrong frame number is picked. This is just an easy way to convert frame numbers from the marker comment to the correct fractional time value for that frame number.

The nearestKey method is exactly that, it is looking for the nearest key and the change happens when you pass the halfway mark. I think nearest key works better than right on the key for this kind of work. I will explain that later.

I did make one error when I wrote the expression. I forgot that frames start at zero. To fix the problem just subtract 1 from the mrkrName like this:

mrkrName = thisComp.layer(index + 1).marker.nearestKey(time).comment;
framesToTime(mrkrName - 1, fps = 1.0 / thisComp.frameDuration)

You don't have to do that if you use zero for the first frame.

As far as the timing issue, that can be fixed by adding a bunch more code but for this kind of work, actually, for timing any animation to audio there is a principal of animation called anticipation. Depending on the motion and depending on the audio, you get a better result if you don't cut right on the beat. I see a lot of people carefully cutting right on the beat of a music track and agonizing over not being able to line a cut with a peak in the waveform display when a more effective cut is usually just before or just after an audio event. Carefully check out Jackie Chan's fight scenes. The punches in the audio track don't ever line up perfectly with the punch on screen. 

This is especially true with Phoneme animations like you are doing. I always have the first frame (1)  as the mouth at rest pose, and I use that marker to slide end words. When adding the audio markers I just put them close to where the sounds start, not right on the exact frame where the "t" sound starts. Then I do a 2 or 3-second preview and drag the sliders around until it looks right. When I added the extra code to the expression to jump to the next phoneme frame exactly at the marker I still had to move things around and it didn't save me any time. 

I don't have the time right now to dig into my archives and share the more complex expression and I'm afraid if I just started typing I would make an error. Dan Ebberts could do it without testing, but I can't.

I hope this helps. 

(EDIT - About an hour later)

I had a few minutes and so I dug through my archives. Here's the code that locks the marker comment to the marker time:

mrkr = thisComp.layer(index + 1).marker;
idx = mrkr.nearestKey(time).index;

if (mrkr.nearestKey(time).time > time){ idx--;}
if (idx < 1) { idx = 1};

mrkrName = mrkr.key(idx).comment

framesToTime(mrkrName - 1, fps = 1.0 / thisComp.frameDuration)

It should really have an else statement added to fix errors but it will put the frame number change right on the same frame as the marker.

[jgermeten]

Thank you again for the detailed response. You're a rock star.

I have actually been working with anticipation for the phonemes, but the problem is that no matter where I place the markers, the phoneme switches at right about 50% between the two markers. Which means if the change is a closed mouth, and four seconds later the character begins a T sound (or the anticipation of it), the mouth changes to the T two seconds too soon. My fix had been to simply create another marker just before the new switch, but I'll tinker with the new code you've given to see if I can skip the hassle. 

Thank you thank you again. 

[Rick Gerard]

Try the second expression.

[jgermeten]

So I have been, and while interestingly it eliminates the need for the slider control, I'm running into some errors, like you seem to have predicted.

I still get some timing drift, but that's manageable, and not a terrible problem. 

What's happening that's completely beyond me is that there's a phoneme that's actually physically drifting to the side. It's only one of the phonemes, and it only happens in certain phoneme orders.

..might be a long shot, but any idea what might be causing that?

[Rick Gerard]

Not without seeing the modified properties of the nested phoneme comp, there should be no keyframes, just sequenced layers all 1 frame long. I'll also need to see the modified properties of the time remapped Phonemes nested comp. You should only have keyframes in time remapping. You might have a position change to line the mouth up, but nothing would be animated.

If you are getting an apparent drift in some of the transitions to different sounds then it may be just the artwork. Going from one frame to another make is look like the mouth is drifting from one side to another.

Like I said, I do all this kind of animation with markers that are labeled with numbers. If that is what you are doing, and you have no keyframes for position, rotation, or scale on any layer then it's an artwork problem. 

[jgermeten]

Again, thanks. 

Maybe it's a bug with After Effects, really. I'm only referring to frames using numbers in the markers, like you said, but sometimes when, say, importing and updating the artwork, the phonemes in the 6-frame sequence (there are 6 phonemes), on the OO sound, for example, it displays the last frame, the L sound. If I duplicate the OO layer it fixes the problem, but once I delete the layer again, it goes back to bugging out. 

¯\_(ツ)_/¯