Calibrate or improve compute lip sync from audio

Report · Aug 13, 2020

Hello,

Does anyone know of a way to calibrate or improve the "Compute lip sync take from scene audio" I'm working with a puppet and after syncing the audio the vismeme's are very inaccurate. I have to completely re-do them which is incredibly time consudming. The audio was taken from a Zoom H6 and is clear without any background noise.

Thanks in advance!

Report · Aug 31, 2020

Hi,

How about tweaking Viseme Detection slider in Preferences -> Lip Sync?

Also, you can switch Lip Sync engine version in Preferences -> Lip Sync -> Version, if the result from the older version of CH with your audio file was better.

Hope this helps.

Report · Sep 03, 2020

Thanks for getting back. I'll give those a try.

Also do you know if there is a way to replace the vismeme "M" with lets say "Uh"? I cant find out how if so.

Report · Sep 03, 2020

Do you mean in a recording? you can right click on viseme and pick a different one. You can also add new ones, split existing ones etc to fine tune the visemes manually.

Report · Sep 03, 2020

I forgot to say right click in a take on the viseme name. To add a new one, click in an empty space of a take

Report · Sep 03, 2020

Yes I know how to manually edit vismeme's in the timeline. But for when I drop in audio tracks and sync with characters I was wondering if there is a way to edit the character rig to replace the vismeme M to be something else like Uh. Because after the vismeme does the sync with character I notice the M vismeme's never look good and I have to manually change to usually Uh.

Report · Sep 03, 2020

You can copy the Uh artwork and name it M if you want to... or as a quick test, remove the M tag from the M layer and add it to the Uh layer. Ch uses the tags, not layer names. But if you reimport the M layer gets autotagged again (so not a good long term strategy).

No other ideas come to mind sorry.

Report · Sep 03, 2020

How about renaming layers ("M" to "Uh", and "Uh" to "M") in the original artwork?

Though this will not change existing tags (M and Uh) in the existing puppet. I need to re-tag as alank99101739 mentioned above.

Or I can import the modified artwork, make the mouth group from the modified artwork to shareable, and replace the original mouth with this shareable one.

If I import the modified artwork, new "M" (contens are "Uh") has the tag "M". But if I added any tags which are not based on the layer name, those tags are not in the puppet from the modified artwork. So that I use the shareable puppet.

Hope this helps.

Report · Sep 16, 2020

I'm trying to figure out how to do this but I just cant figure it out. I've found all the vismeme layers in the rig tab under the puppet but i cant figure out out to replace the character's M triggers with Uh triggers. I know this is now a good long term strategy but when I manually change M to Uh it always looks so much better.

Could you possibly walk me through how to make the M trigger be Uh and also keep the Uh trigger as Uh. Hope this makes sense.

thanks in advance

Report · Sep 16, 2020

One approach is to use "edit original" to open up the artwork file (photoshop or illustrator) and then find the viseme layers in the artwork. To make a layer be both "m" and "uh" rename the layer/group to be "m uh". Then find the other layer you don't want and delete it (or rename it to "unwanted" - Just make sure it does not have the word "m" or "uh" in it.) Save the file and go back the Character Animator.

What should happen is the renamed layer will get both the M and Uh tags added (check in the panel to the right that shows the tags and select teh "m uh" layer). A layer can have multiple tags - and if you separate the tag names with spaces i think it will add them all.

Report · Sep 18, 2020

that worked! thanks for the walkthough on doing this. I'll see if this will work out for a permanent/long term solution.