Isolating Lips from lip-sync video

Question

Greetings, does anyone have tips on how to isolating lips from lip-sync video?

Final goal: is to isolate the pair of lips and delete the rest of the video so that I can apply the lips to a different background-Preferably the isolation would be hard edge cut with an option to increase surface area so it blurs a bit into the surrounding skin.

I have tried so many motion tracking techniques its making my head spin, all of them end up losing track and leaving me to do very slow progress in individual frames

Here is a video of what I'm working with...

YouTube

I'm thinking of doing a rerecording this time around by either-

0) asking model to keep face completely still this time

1)Painting the skin around lips green

2)putting small stickers on lips to use as anchors for motion tracking (but then how do I remove these stickers post production for smooth lip color?)

I think the reason why none of these motion trackers or tutorials on it have worked is because the lips (when producing speech) is an object that morphs in all kind of strange ways that usual motion tracking application doesn't.

Thank you for your time!

Rick Gerard · Accepted Answer

You are not approaching your problem rationally. The tip of the nose and the bridge of the nose don't move when someone is speaking. That's what you track. You could also use the corners of the eyes. Now the face is locked in the frame and you mask around the mouth. Trying to track the entire mouth can be done but not with the tools that come with After Effects. Given proper photography you may be able to track and roto the mouth and lips in Mocha but it would take some preparation and some careful camera work. Even using Mocha, you would want to Stabilize the shot first using parts of the face that don't move.

Once you have the face stabilized, not tracked but stabilized, then you can either choose s soft edge mask and use makeup to blend the surrounding area of the mouth with the other shot, rotoscope the edges of the mouth to get a tighter mask, or use makeup and keying to cut out just the mouth and lips.

Mylenium · Answer

And have you actually considered just using good old-fashioned rotoscoping - by hand using masks and what have you? Sorry, but I don't even know what tracking has anything to do with anything here - unless you really mean using mocha Pro or other advanced tools. The rest is merely you looking for simple solutions. Yes, roto work is tedious, but sometimes unavoidable. There isn't a magic button for everything.

Mylenium

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded