Monty Python style mouth only animation won't lip sync to recorded audio! So simple a concept, yet-!

Report · Oct 14, 2019

Hi everybody. I'm a noob trying to make a simple-a__ switch layer animation. It could not be simpler yet, I have agonized for more than a week trying to get my litter critter to speak but to my dismay, no cigar.

To be clear- I have all the heads (mouthing the various sounds) all ready and on seperate layers (from Photoshop) and called myself rigging it. Maybe I didn't. Hard to tell as there is no real simple ' "A" is clicked to get "B" result ' type videos on such a simple level. I think what I want is too simple. I don't want it to look good. I want it crappy Monty Python style Terry Gilliam like.

I did everything I know (which prob ain't much) and set it up in layers and everything- even followed the examples from Ok Samuri but nothing works: I made different heads with the various mouth positions- I matched them to the Vsimes (wrong spelling I bet) and clicked the head and got a blue outline- I did the scan to compute the lip sync- I brought in my pre-recorded audio track I want it to sync to- I added plus signs, subtracted plus signs and devided it all by insanity squared. I friggin' clicked every menu in Character CC and still my muppet is mute.

Help meeeeeee---!

Report · Oct 14, 2019

There are two ways to do mouths ... visemes where you replace a default mouth with a different mouth image for different visemes, and nutcracker jaw which slides a jaw up and down. Nutcracker might visually look more like what you want, but I don't think can be computed from and audio track as it is controlled by the webcam. https://helpx.adobe.com/adobe-character-animator/using/behavior.html#Nutcracker

Assuming you are using the default visemes, the hierarchy of the layers in the artwork file are very important for ch. Could you share a screenshot of the rigging hierarchy you have for your puppet? eg do you have a Head layer containing a Mouth layer with visemes under that? There are special names for the visemes (Ah, Nueutral, Smile etc). The Face behavior looks for the mouth layer and expects the visemes as immediate children, so if you get those names wrong it won't work.

If you want a video to look at, try episode 22 from https://extra-ordinary.tv/2018/10/22/project-wookie-a-beginner-youtube-playlist/. There may be better ones around, but i know where this one is lol!

But i suspect you might want to use nutcracker jaw, which i have never used myself.

Report · Oct 15, 2019

Just first wanted to thank you for the fast response. It is truly appreciated and my gratitude cannot be overstated. Now- I included a few screenshots of my PhotoShop file and hierarchy. I also, for good measure, included my Charachter Animator screen. I did because maybe something should be armed with the red dot. Maybe not. There could be any number of problems but I don't see any in that program. Another thing is that my visemes are there- custom mouths and everything, but they are attached to the non-moving head. Giving the illusion, kinda of a nutcracker. To be precise, what I'm doing isn't a nutcracker at all. I will or intend to, use the multi-mouth visemes. Thanks again! I know its something small and stupid! G7

Report · Oct 15, 2019

hi, sorry, the screenshots did not seem to have come through. The rigging panel showing the puppet hierarchy I would like to see in particular

Report · Oct 19, 2019

Report · Oct 19, 2019

Got the screenshots that time, thanks! I don't know if this will make a difference, but...

The Neutral layer should be under Mouth as well - looks like it is up one layer. I normally make it first and it is the one layer that is visible by default.

I also don't put "+" on all the mouth positions. I will put that on the Mouth layer so the mouth is independent. I don't know if it will make a difference, but just looking for differences in what you have.

There might also be a space after the "+" characters? Leave that space out to be safe. But the panel showing the linkages is perfect. You can see it found the Mouth layer but not the sounds under it. I would try removing the "+" for each mouth layer and see if that fixes it (and moving the Neutral layer under Mouth).

If that does not work, you can click on a layer then over on the right is a "tags" section of the layer properties. CH will auto tag layers based on names, but it is the tags that really matter. E.g. the Neutral layer should have the Neutral tag on it. If it does not, then you can add it manually. Behaviors actually use the tag names, not the layer names, to find the mouth positions. Its just the tags get added automatically if the layer names are correct.

Report · Oct 19, 2019

(If you look at the linkages screenshot, you can see a "1" for the mouth layer and zeros for all the others - this indicates it found the mouth but no mouth positions)

Report · Oct 21, 2019

Thank you. I will look into your suggestions and let you know as soon as I do!

Report · Oct 24, 2019

Gary sent me a direct message with the puppet. I tried to reply directly, but it said I had sent too many private messages. So replying here.

Hi Gary, it looks like I have a newer version of CH installed so I don't think I can send you the puppet (might not work), so here are instructions. To just get the mouth working its not much work - you are very close.

In Photoshop, drag the Neutral layer and all the visemes into the "Mouth" group. Currently they are not in the mouth group, they are next to it. This is the primary problem.
Rename "Mouth" to "+Mouth".
Rename "+ Neutral" etc for all the visemes to just "Neutral" (remove all the "+" characters - use it on Mouth instead)

You can tell all the visemes are under Mouth by clicking on the little expland/collapse triangle next to the "Mouth" layer name in Photoshop. If collapsing Mouth you can still see visemes, those visemes are not inside the group.

Once you have done that, lipsync started working for me!

If you want to go further, I suggest some more changes if you want the head to move with the webcam.

Click on the "DOG" layer in rigging mode. Over on the right expand the "Tags" section of the properties. In my copy the "DOG" layer has had the "Head" tag added. Remove that tag. Note: There is a little "A" icon in the top right corner. If you click that it brings up the text names instead of pretty graphics. I find that mode easier. "DOG" should not have any tags on it. This "Head" tag on the "DOG" layer breaks the Face behavior.
I suggest you rename the root "DOG" layer to "+Dog" Photoshop - make the root independent (it generally works better).
Rename "Head" to "+Head" in Photosohp. I am assuming you want head tilts without body moving too.
Add the "Face" behavior back to the puppet root - looks like you removed all the behaviors except 2.
Expand the "Face" behavior properties and look for "Head Position Strength" - this is 100% by default. Make it 0%. (You can try 10% if you like.) This stops the head "drifting" away from the body.
Click on the LipSync behavior. If you look in the properties panel you will see there are some missing visemes. Decide if you want to add these in as well. (You can copy other existing layers that are close.) I think it will default to Neutral if it wants to use that sound and there is no viseme.

With that the head should tilt from side to side while talking!

Report · May 14, 2023

This is so late you may be dead but-

Thank You!