Swap Faces (Even when they are Turning)

[faizanshaikh.net]

I'm working on small movie clip where the actor is frequently moving his face backwards or on sideways in such a way that his neck is visible. I tracked using Mocha, applied the tracked data to a null object and pick whipped the image to null but the image still faces front.

I understand that ideally the image will not rotate along with the face so I splitted and created another layer to work on when the actor is moving his head and used replacement image which has a turned head but still the accuracy is low when the actor completely rotates his face. I am looking for an option where I can give more accuracy when the faces are turned. Is there any way to achieve this?

[angie_taylor]

In order to understand what you're tryig to do we would need to see an example as your descriptions doesnt tell us enough. Can you screen capture what you are attempting then post the video please?

[Rick Gerard]

You will need to do corner pin tracking on the main plane of the face. Once that is done, depending on the shot, you have two options. One is to add the corner pin data to a solid with a grid and then run camera tracking on only the moving solid layer, the other would be to stabilize the corner pin track using CC Power pin and 4 expressions to get a face plane that distorts but does not move. The last step would be to replace the solid or the stabilized corner pin of the face with a new face. 

Show us the shot and we can give you a better idea of the most efficient approach.

[faizanshaikh.net]

That seems a tough one, haha. How about the face tracker with puppet pins? is it going to achieve the same results?

[Mathias Moehl]

If you use a null for your tracking data, the image will only move, scale and rotate in 2D, but never deform in 3D perspective. You need a corner pin or stabilized precomp to get the perspective (shear) information.

See this tutorial for details:

https://mamoworld.com/tutorials/keep-current-frame-why-never-use-nulls-move-layers-mocha-track-mi-v6...

If the head is turning enough, even the perspective track will not give you 100% accurate results, since the face is not a flat surface. Maybe adding a warp manually inside a stabilized precomp might help - I described this in this pretty old tutorial in detail: https://www.youtube.com/watch?v=IGtfIPJyrnc&feature=emb_logo

If this still is not good enough, you really need a full 3D solution where you create a 3D geometry of your character and project the new image to it. But this is way more work and rather a job for NUKE in combination with some 3D modelling app and not for Ae's "flat 2D layers in 3D space" approach.