Visemes from live recording are not good, but try this...

New Here ,
Oct 12, 2020 Oct 12, 2020

Copy link to clipboard

Copied

I've been doing some recording and struggle with the visemes, they always seem out, or missing.

 

The Lip Sync Behavior was selected and the camera seemed to be working fine.

 

I recorded this; A is for apple. I know what you are thinking, another Shakespeare 🙂

 

Well, the Viseme result were really bad, so I dicided to try something.

 

I clicked on the audio take in the timeline.

 

Then selected, Compute Lip Sync Take from Scene audio.     Menu: Timeline / Compute Lip Sync Take from Scene audio

 

The results are amazing. Not brilliat, but a whole let better and cetainly useable!

 

My theory is that when doing a live recording, the visemes are created via the mouth detection from the camera and not related to live voice audio. Therefore if the Camera is not picking up well, the viseme results will be poor too.

The algorithm used in, Compute Lip Sync Take from Scene Audio,  seems to produces better viseme output.

So if recording live, rather than pulling in audio file, I would suggest using the Compute Lip Sync option to improve results before editing. Should save time and effort.

 

Please try this out and let me know if you find the same thing, or if this is just me?

TOPICS
Tips and tricks

Views

57

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Oct 12, 2020 Oct 12, 2020

Copy link to clipboard

Copied

LATEST

Interesting! Thanks for sharing. I wonder if doing it live it has trouble keeping up with CPU trying to do everything real time, where as "Compute from audio" it can take as much time as it likes to do a good job (and does not have to guarantee real-time results). I suspect its the same code, but live it MUST come up with an answer in a certain period of time, where as compute from lip sync does not have that restriction. Sort of like if you have a fixed time to complete an exam or not.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines