Visemes from live recording are not good, but try this...

Question

I've been doing some recording and struggle with the visemes, they always seem out, or missing.

The Lip Sync Behavior was selected and the camera seemed to be working fine.

I recorded this; A is for apple. I know what you are thinking, another Shakespeare 🙂

Well, the Viseme result were really bad, so I dicided to try something.

I clicked on the audio take in the timeline.

Then selected, Compute Lip Sync Take from Scene audio. Menu: Timeline / Compute Lip Sync Take from Scene audio

The results are amazing. Not brilliat, but a whole let better and cetainly useable!

My theory is that when doing a live recording, the visemes are created via the mouth detection from the camera and not related to live voice audio. Therefore if the Camera is not picking up well, the viseme results will be poor too.

The algorithm used in, Compute Lip Sync Take from Scene Audio, seems to produces better viseme output.

So if recording live, rather than pulling in audio file, I would suggest using the Compute Lip Sync option to improve results before editing. Should save time and effort.

Please try this out and let me know if you find the same thing, or if this is just me?

alank99101739 · Answer

Interesting! Thanks for sharing. I wonder if doing it live it has trouble keeping up with CPU trying to do everything real time, where as "Compute from audio" it can take as much time as it likes to do a good job (and does not have to guarantee real-time results). I suspect its the same code, but live it MUST come up with an answer in a certain period of time, where as compute from lip sync does not have that restriction. Sort of like if you have a fixed time to complete an exam or not.

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded