Skip to main content
Jeff Almasol
Adobe Employee
Jeff Almasol
Adobe Employee
August 14, 2020
Question

Feature Focus: More accurate lip sync

  • August 14, 2020
  • 2 replies
  • 2536 views

The latest version (3.4) of the lip sync engine improves automatic lip sync and timing of mouth shapes (visemes). Prior versions can still be selected and viseme detection can be tuned in Lip Sync preferences.

 

 

Try out the improved lip sync engine in both live/rehearsal and offline (Timeline > Compute Lip Sync Take from Scene Audio) workflows, and let us know how it's working for you.

 

Are you needing to do fewer corrections to computed lip sync?

 

Are you able to adjust the Viseme Detection and Audio-based Muting settings (in Lip Sync preferences) to get desired results?

 

 

(Use this Beta forum thread to discuss this feature and share your feedback with the Character Animator team and other Beta users. If you encounter a bug while using this feature, let us know by posting a reply here or choosing "Report a bug" from the "Provide feedback" icon in the top-right corner of the app.)

This topic has been closed for replies.

2 replies

Timm Schindel
Timm Schindel
Participating Frequently
August 28, 2020

I'm under the impression that it has improved. At least in the test project I use. It's 8fps and has a reduced amount of visemes.

Frannie@Cartoonfun
Frannie@Cartoonfun
Known Participant
August 23, 2020

Having run a few tests comparing Ch Beta generation of lip-sync, with existing offline recorded speech tracks that I've used previously with Ch 2020, it feels like the amount of post-compute correction that I need to do in Ch Beta is pretty much the same.

 

Pre-existing inaccuracies ('quirks' 🙂) are extant. For example, I find that in both Ch 2020 and Ch Beta, 'S' is almost always mistakenly generated when it needs to be an 'F' ... and 'D' or 'L' is generated when it should be an 'M".

It may well be down to my diction, of course... a slight Cornish (Cornwall, UK) burr as opposed to Received Pronunciation.

Anyway, hope the feedback helps (loving the new Beta features overall 🙂)

    Jeff Almasol
    Adobe Employee
    Jeff AlmasolAuthor
    Adobe Employee
    August 24, 2020

    Hi @Frannie,

     

    Would you be willing to share one or more sample audio files that are not working well, and that our research team can diagnose internally? If so, you can send me a direct message with a link to a shared archive (via Dropbox, Google Drive, etc.).

     

    Also, were there any settings in the Lip Sync preferences that improved the results for you?

     

    Thanks.

    ebenheizerm28787053
    ebenheizerm28787053
    Inspiring
    August 25, 2020

    Is this lip sync optimized only for english? I am making an Indonesian animation.

    Terms & ConditionsAccessibility statement
    Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices