Lip Sync From Scene Audio Not Working

[Amanda_Armstrong]

Hello!

I can't get lip sync from scene audio to work on one of my puppets. The track works fine for other puppets, and I've tried multiple tracks, but I get nothing. No visemes at all. My puppet is extremely complicated, so perhaps that's the problem? I have 28 dangles on each of my 7 head turns. Lip sync seems to be working with microphone input, though there is quite a bit of lag. Am I just asking too much?

I'm happy to upload my puppet if need-be.

[Dan Ramirez]

Hard to tell without seeing the puppet, but the complexity of the puppet shouldn't matter. Make sure the right puppet track is selected and make sure that the lip sync behavior is record armed in the properties panel(red button). If that doesn't work, if you post the file, we can debug it ASAP.

-Dan R.

[Dan Ramirez]

Awesome puppet!

I can also reproduce your problem which is surprising to me. I'll get back to you ASAP. Thanks!

[Dan Ramirez · Accepted solution]

OK. Found the problem and we've already fixed this bug in our internal builds. There are 196 instances of the dangle behavior on this puppet because each piece of fur(28) on each view(7) is an independent layer with a dangle behavior applied. While a bit inefficient, this shouldn't be a problem, but we had a bug where the lip sync compute process was being impacted by the complexity of the puppet. We've since fixed that bug. We did find a workaround that also worked, turn off your microphone and camera before doing "Compute Lip Sync from Scene Audio".

Dan R.

CH QA

[Dan Ramirez]

By the way, the best structure is likely one dangle behavior on each view(7) instead of a dangle behavior for each dangle handle. This would give you individual control over the way each of the 7 views responds to gravity. Right now, you have individual control over each piece of fur in every view, which is overkill.

Dan R.

CH QA

[Amanda_Armstrong]

Hi Dan,

I tried turning off camera and mic like you suggested, and I still can't get it to compute. It goes through the motions and makes a track for visemes, but the track is still empty. I also removed all of the dangle behaviors (left the dangle handles in place) and that didn't seem to help either. I sent you a message with a link to the updated puppet just in case.

Were y'all able to get it to work on your end with the workaround?

Thank you so much for your help and the tip on dangles- I can see that that was total overkill now.

Here are my settings with my last attempt to compute the audio.

[Dan Ramirez]

https://adobe.ly/2qXrpVghttp://

I made a few edits to how the dangle behaviors were structured. This seems to compute lip sync fine for me.
Dan R.
CH QA

[Amanda_Armstrong]

That link isn't working for me. Can you repost?

[Dan Ramirez]

Link to puppet file

Try this link. The previous link has "http://" appended to it. I tried to edit the post, but the forum is having some issues right now.

[Amanda_Armstrong]

Good morning!

I downloaded your edit and I'm still having the same problem. I can't for the life of me figure out what's going on!

[Dan Ramirez]

email me directly at dramirez|at|adobe.com and we'll get you into the private beta program.

Dan R.

CH QA

[dianes68801335]

Hello I have a similar issue.  I have attached the compute lip sync to the right puppet and the visemes show up in the time line but the puppets mouth does not move. Web cam and mic are off.

Diane

[Amanda_Armstrong]

Hi Diane,

I think the issue might be that you actually want to have visibility of mouth shapes turned on. I finally got mine to work by disarming everything but Lip Sync before I computed. My puppet was overly complicated, and that seem to have been my issue.

Try turning on the eyeballs for your mouth shapes and see if that works!

[dianes68801335]

Tried that but it it didn't work.

[dianes68801335]

thanks for the suggestion.

[Amanda_Armstrong]

Hmmm...Do you have your mouth group tagged as "Mouth" or as "Mouth Group"? If it's currently only tagged as "Mouth" that could be the problem.

Do your Neutral, Smile and Surprise work as expected?

[dianes68801335]

Thank You!!!!! Sending you a big ol cyber smack on the lips.  For some reason most of my puppets get automatically tagged and this one had to have them all done individually.

Really appreciate your help