Skip to main content
jamesf16266414
jamesf16266414
Participating Frequently
March 27, 2019
Answered

Mouth disappearing for a frame in between lip sync motions - fix?

  • March 27, 2019
  • 5 replies
  • 4785 views

Hey all,

So I've got some custom puppets set up as PSDs based on the example Blank Face included in the program. All's working well, except for one thing - when lip-syncing the puppet to a pre-recorded audio clip (using the Timeline command: Compute Lip Sync from Scene Audio).


Which works great, except for one issue: fairly frequently, between changing shapes or during pauses, the mouth will disappear for a single frame. Which is fairly noticeable as it keeps happening and makes the animation pretty painful to look at.

Any fixes? I've scoured google and the forums with no luck.

Thanks in advance!

    This topic has been closed for replies.
    Correct answer DanTull

    Update: so I tried re-computing the lip sync with the new cycle layers behavior added to each puppet's grouped visemes. It looks like the mouth is still disappearing on those grouped visemes. Is there a specific setting I should change?


    What's really strange to me is that I'll go frame-by-frame in one direction and see the mouth, and the frame will show up before I let go out of the mouse - then, when I do, the mouth disappears again, even though the cursor line is still on the same frame.


    If you import Blank Face and select one of its visemes with a group and a cycle layers behavior, there's an option in the Edit menu called "Copy Behaviors", then you can select each of your mouth groups (Aa, Uhh, and W-Oo) and paste the cycle layers behavior on each.

    Assuming those layers have mouths on them, that should work.

    Another way you could confirm is by turning on the eyeball for the first layer in each of those groups. If I'm right, that should fix missing mouth problem. The mouths won't animate when sustained, but they should at least show up.

    DT

    5 replies

    alank99101739
    alank99101739
    Legend
    March 28, 2019

    That last screenshot, the cursor was not on the last frame of the sequence. Could you shows a few consecutive frames showing mouth appearing and disappearing?

    Oh, and do you have profiles or anything else unusual? Starting to wonder if its not the visemes, but rather something else in the puppet switching something causing the mouth to disappear. To test that, if you stretch and slide the visemes around manuall, keeping the blue playhead on the same frame, does the mouth appear? Then if you pick a position where the mouth is visible and drag things around there again, does it disappear? If no change, that would be stronger evidence its something other than the mouth getting in the way.

    jamesf16266414
    jamesf16266414Author
    Participating Frequently
    March 29, 2019

    Hey Alan,

    Not using any profile shots or anything too strange - I tried to stick to the Blank Face format as much as possible because it's my first time using the software. In the frames where the mouth is gone, the mouth does not appear regardless of how I play around with the visemes manually. Similarly, adjusting visemes where the mouth is visible doesn't make it disappear, it just changes the shape based on which viseme takes over for that frame.

    Here are some shots of the frame timeline near the end of the scene:

    Hope these help! See my reply to Dan for the Rigging breakdown.

    - James

    DanTull
    Adobe Employee
    DanTull
    Adobe Employee
    March 29, 2019

    Both of the missing mouths are on groups vs single layers. If you click those groups do they have a cycle layers behavior added? What are the settings for that cycle? My current bet is that they either don't or have settings for the cycle that result in frames not being shown for the whole time that the mouth triggers the group (missing hold on last layer or set to immediately instead of when triggered, etc).

    Thanks -- DT

    alank99101739
    alank99101739
    Legend
    March 28, 2019

    Do you use cycle layers at all?

    Could you include a screen shot of the rigging hierarchy for the puppet, expanding the mouth in particular?

    DanTull​ - any thoughts on this one Dan?

    alank99101739
    alank99101739
    Legend
    March 28, 2019

    (If you can fix one, I would just recompute from lipsync again - faster than adjusting them all one by one)

    jamesf16266414
    jamesf16266414Author
    Participating Frequently
    March 28, 2019

    Thank you for the advice,

    I didn't manually change the frame rate - could it be related to the audio file I'm using?

    I dragged a few of the visemes into new positions at the frame divides and then recomputed the lip sync, but it just put them back to their original positions. I tried re-aligning some of them manually and replaying the video, just to see if it would fix it, but it doesn't seem to have: picture attached.

    Let me know if you have any more thoughts,

    James

    DanTull
    Adobe Employee
    DanTull
    Adobe Employee
    March 28, 2019

    The lip sync engine actually operates at 80hz so its outputs are not necessarily aligned to frame boundaries. For some framerates, it's even possible for it to generate a viseme in an interframe gap that is never displayed. The lip sync behavior just takes the value that's active at the middle of the frame.

    We've considered whether we might get better results in some cases by informing it of the target rate, but haven't done those experiments yet.

    All that is to say that I don't think the misaligned visemes is the key part.

    I'd be curious to see the puppet's mouth rigging. Visemes being mutually exclusive, it seems more likely that one of them is a cycle layers animation (like what Alan asked about above) that is finishing and running out frames (instead of being set to hold on last layer) or a mistagged item causing one of the mouth shapes to be missing.

    If you look at the properties panel for the lipsync behavior on the puppet, there's a section that lists which layers matched for each viseme. Maybe that will provide clues as to what is wrong?

    DT

    alank99101739
    alank99101739
    Legend
    March 28, 2019

    Interesting! Looking at the boundaries of the visemes they don’t seem to line up exactly with frames. Did you change the frame rate? My suggestion is to try dragging the edges of the visemes to line up with the frame boundaries and see if that helps.  E.g. the Aa viseme ends just before the end of the frame. Maybe that is the problem?

    alank99101739
    alank99101739
    Legend
    March 27, 2019

    Could you zoom in on the timeline and screen grab the visemes that compute from  lipsync generated? The zoom control is in the bottom right corner (slider between two pictures of mountains or something). Can you move the playhead to the position where the frame disappears - is there a viseme there for that frame? If you can screenshot that and paste it here in the forums that would be great.

    My guess is a viseme is broken (not tagged correctly - e.g. spelling error in name) so when that viseme is displayed nothing shows up. But if that is the case, you should be able to zoom in on the frame where everything disappears and see the viseme name that is broken.

    jamesf16266414
    jamesf16266414Author
    Participating Frequently
    March 28, 2019

    Hi Alan, thanks for getting in touch.

    Here's a screenshot of the kind of thing I'm talking about:

    Let me know if that helps or what you can make of it. Thanks in advance!

    Terms & ConditionsAccessibility statement
    Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices