Skip to main content
I
Il Cacciatore di Tool
Participant
September 23, 2021
Question

Audio Sync + Transcript

  • September 23, 2021
  • 1 reply
  • 624 views

Hi Dave, hi community.
There's an issue in new audio+transcript labial sync.
Note: I used an Italian audio (1 minute long) so .SRT is in Italian.
If I try the syncing with audio only, all the audio file is synced but when I use the audio+transcript, the end of the audio file (10 seconds) is not synced and no visemes appear in the timeline.

It's a strange behavior because only the last 10 second are not synced, and the result of the first 50 seconds is drastically better than the sync of audio only.
Have a nice day.

This topic has been closed for replies.

1 reply

CoSA_DaveS
Adobe Employee
CoSA_DaveS
Adobe Employee
September 24, 2021

Thank you for the report. Adobe is closed today (Friday); we'll get back to you next week.

 

Do any markers with error info appear in the timeline?

I
Il Cacciatore di ToolAuthor
Participant
September 27, 2021

Hi.
No markers with error in the timeline.
I tried again yesterday. Same result.

Have a nice day.

D
Community Manager
dtull-adobeCommunity Manager
Community Manager
September 28, 2021

Hi,

 

Thanks for giving it a try! I think this may be a similar issue to what I described in a response here: https://community.adobe.com/t5/character-animator-beta-discussions/feature-focus-transcript-based-lip-sync/m-p/12407907#M452

The code tries to align sounds to phonemes and sometimes it does that a bit to aggressively so it runs out of text before it runs out of audio. It is possible this is exacerbated by it being Italian, though it does happen sometimes even for English.


One thing I might have to check is how the backend handles accented characters. If it is skipping them instead of mapping them to an unaccented form (would have to go digging in the code to confirm), it might make the effect worse. One way to test that would be to replace such characters with unaccented ones and see how the result compares.

 

If you'd like to share the audio and transcript, I can try to take a closer look, but I think I have a rough idea of what's going wrong. Thanks for the report! It's good to know that even on a language we haven't tested, there's at least some promising results (even if there's also some issues to sort out)!

 

Dan Tull

Adobe Character Animator Team

 

Terms & ConditionsAccessibility statement
Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices