Lip sync for puppet with profiles

[Alex.15]

Hi,

I have a puppet (rigged with the basics for being able to reuse it) that has head profiles and swap faces in place and when I compute lip sync from a new audio file it computes 10 takes of visemes, which in turn are very hard to manipulate when I want to make a change. Am I going wrong somewhere? There must be an easier way to have just one take of visemes from audio and have the puppet lip syncing when turning head to profiles too. Because I`ve tried just adding lip sync behaviour to frontal only with one take of visemes, but then when I need profile view lip sync does not work.

Any suggestions to make this easier to work with?

Much appreciated in advance,

Alex

[Platty Puss]

Hi Alex.

    Hmmmmm, do you think you might want to start with a less complicated project? I can only speak about my experience with head turning behavior and or swap sets with head turns. I discovered that if I lay down the audio track first and create the visimes, that when I go back over the puppet with head turning or swap sets then it works just  fine. Laying down the audio track after having had put down the head turns, for me, was sometimes problimatic.

     I am confused as to how you managed to get all those visimes at once.

     If you continue to have problems I would suggest scaling back your project and start with fewer head and face objects.

    Keep us posted as you move along.

Be well fellow pupeteer - Platty

[alank99101739]

Could it be there is a separate LipSync behavior per profile? Or per profile and mouth expression group?

I dont remember if this is needed - I would have hoped one LipSync behavior would work across all profiles and all facial expressions at the same time, so only needed one set of visemes. But don't have time to check just now.

[Alex.15]

I had it added per each profile/swap, hence i had 10 takes computed, because otherwise it woudln`t work. I`ve tried taking them all off and just keeping the puppet auto-added one, it doesn`t work at all, nothing. then I`ve tried with adding lip sync just for frontal, but then it doesn`t work for profiles. 

Anyhow I may be doing something wrong.. But hey thanks!

[Alex.15]

Thanks Platty. I know I`ve got myself into a bit of a complicated thing, but it`s what I thought it would help to have the basics rigging done, so I wouldn`t have to redo it everytime for the same puppet. I am with you it is problematic indeed but I`d rather try to make it work than redoing the same thing time after time. I`ve done a couple by now and as you were mentioning when I didn`t have it rigged with headturner upfront, it worked. Now I am trying to find out if there is a way to go about this without having to start from scratch every time if it makes sense? 

Aww you too pupeteer, Keep safe x

[Platty Puss]

Thanks Alex. This is a curious problem. Please keep us informed. I would love to see the end result.

Platty

[ElectroPuppet]

I've built many puppets with head turner and they all only ever generate one track of lip sync data. So you have done something outside the box there! Hard to diagnose without seeing a screenshot - but I'll race through some things to check.

For head turner you should have a layer/folder called 'Head' inside your puppet. Then nested in that folder are folders called;

+left profile
+left quarter
+frontal
+right quarter
+right profile

Inside each of those should be your mouth sets. Character Animator should automatically pick up each head profile tag when you apply the head turner to the 'Head' folder. 

Then inside each head profile you have your eyes, mouth folders.

If you start adding other things, like different body views or the ability to walk, the rigging can get much more complicated. There are a number of free puppets or paid puppets that would be worth getting and studying the structure - as finding this out after you've spent all that time creating and rigging can be frustrating! (We have a 'Free Dudley' puppet and various paid puppets you can reference @ ElectroPuppet.com that all do what you are after). Hope that helps Alex.

[Alex.15]

Heyy, thanks for your answer.

Yes I did do something out of the box, cause otherwise the mouth swap set won`t work, when the puppet has only one lip sync behaviour at the puppet level, as it should be normally, only the default set works, the sad set won`t work, it shows that it`s changing to the sad set but not lip syncing, it solely shows the mouth shapes there. I`ve tried everything, but only adding a new lip sync behaviour to the frontal/mouth group makes it work, hence if i added a lipsync behaviour to each mouth/ profiles & swap faces I`ve ended up with 10 compute lip syncs, does this make more sense?

Hierarchy I think it should be fine but you have a look maybe can spot something, also all the other triggers work just fine, it`s just the mouth set that I can`t get around to make it work without adding the additional lip behaviour, and that`s actually keeping me stuck with this.

See the screen shot below and please do let me know if you know of anything that might work in this case?! 

Much appreciated

Alex

[alank99101739]

I think the visemes have to be immediate children of Mouth. I often do:

- Gobs (slang for mouth)

- - Sad Mouth (space between words so gets tagged as mouth)

- - - Neutral

- - - Ah

- - - etc

- - Happy Mouth (space between words)

- - - Neutral

- - - Surprised

- - - Ah

- - - etc

Then I put a swapset on "Gobs". I was going to check up on using one LipSync behavior on multiple mouths, but CH refused to start for some reason - maybe time for a reboot! I look at Face and LibSync etc face views/handles/etc to see what they bound through to as well. E.g. did it bind to both Mouth layers? Then the visemes under the mouth layers?

[Alex.15]

But it`s what I`ve done as well..as you can see both mouths are under the parent layer of mouth, each mouth set has the visemes straight under. And I`ve added the swapset for Mouth parent layer for both sad mouth and happy mouth, but lipsync still only works for happy mouth. I`ve only renamed them differently as sad set and happy set, but I`d think it wouldn`t be a problem the name as the happy mouth works? and they are tagged properly both of them as mouth...as I was mentioning the sad set appears when the trigger is on but does not lipsync, it only shows the puppet with all the sad mouth shapes on it. 

[ElectroPuppet]

Hi Alex,

I just had a quick look at a puppet I created that has different mouth states. I have a top level folder called 'MouthStates' and then under that I have different folders;

-Frown
-Terror
-Mouth

'Frown' contains sad mouths, 'Terror' is a triggered scream and 'Mouth' contains the normal mouths (what you have called 'Happy Mouth') I don't have time to look into this right now - however my guess is you need to have a default 'Mouth' group that Character Animator recognises. Perhaps try changing 'Happy Mouths' to 'Mouth' and make sure your sad mouth also has the mouth tag. Let us know how you go.

[ElectroPuppet]

Oh - and change that top level 'Mouth' tag to something like 'MouthStates'.

-MouthStates
   - Mouth
   -SadMouth

[Alex.15]

Thanks but do you think changing some words would make a difference if everything is tagged correctly and the default mouth working even if tagged happy mouth and not just mouth, it makes little sense to me.