3d camera render

[garrobal]

Hello.  I'm a bit new with after effects and have a question.  I am trying to track and replace a painting on a wall with my own work in a video.  There is major movement and major scale changes.  The painting also becomes occluded  for a lot, most of, the video.  When I use the tracker it seems that I can adjust the solid successfully for parts but then it drifts dramatically in other parts.  I was wondering if there is a way to split the video into segments, then 3d camera track each segment then string them back together somehow so that there is no large changes in the image from one part to the next.  I will try to attach the video clip in question.  If sucessful it is the painting on the right just inside the doorway.  Also, if it makes a difference I do not want to replace the frame of the painting just the painting.  Thanks for your time.  Regards  Gary

PS I do not see a way to attach the video as I can only find where it says to use a url, unless I am mistaking.

[Rick Gerard]

You can post a link to online storage like dropbox or google drive.

From your description I think you will probably be better off using Mocha AE and corenr pin tracking or even some advanced  corner pin tracking to do your composite. 

Camera tracking requires a lens with very little distortion, a fair amount of fixed geometry, some kind of a ground plane, and significant perspective and paralax changes to work well. If you don't have that you end up with a track that thinks it is just a camera pan so there is no real depth information. On the other hand, if you shot is compatible with Camera Tracking then it is very easy to just split the layer into segments and add a camera and reference solid or null to the shot as long as the object you want to replace becomes at least mostly hidden so you can cut between the diffeent cameras you will add t the comp. 

If you can post your video on YouTube or Vimeo or load it up in a file sharing locaion we will have a much better xhance of directing you to an efficien workflow.

[garrobal]

The painting I'm trying to track is the one on the right immediately after going through the doorway and thanks for your reply.  I spent days trying to get it to work with mocha with partial success.  I know how to use the distort filter corner pin but do not know about corner pin tracking.  Can you recomend a good youtube video for it.  Thanks a lot.  Regards, Gary

[Rick Gerard]

I took a look at your shot, made a screen recording and I'm going to give you the basics. First, let me talk about the problems with the shot. The first problem that I see is the shallow depth of field. You are going to have to add some kind of a blur to the replacement image to match the look. Camera Lens Blur is the best option that comes with AE. The second problem is that the picture frame has an edge that covers up the right side of the image. This can turn out to be a very tedious roto job.  

Let's look at the Mocha Setup. Here's how Mocha AE opens up and where you should start:

You are going to have to start on this frame where most of the picture is occluded by the door frame. You'll also have to track backward, then jump to the original keyframe and track forwards. Perspective tracking must be enabled.

The next step is to view the surface and make sure that it fits the picture frame. The track is not going to be perfect so you will have to change the Mocha workspace to Classic and adjust the surface so that it stays lined up with the frame. Adding a grid to the layer helps. You should be able to fix the track enough with just a few keyframes. That looks like this:

Then you save and return to After Effects. For a job like this, I always add a placeholder using a comp sized shape layer that has a color that will let me see the background using a blend mode. The first step is to import the corner pin data to the footage layer using Mocha, then apply the corner pin data to the placeholder layer. That looks like this:

I haven't switched the blend mode on the placeholder yet.

The next step would be to start a new shape layer to be used as a track matte for the doorway edge. I used the pen tool, trimmed the shape layer to just be long enough to cover the part of the shot where the doorway covered the painting. It took quite a few keyframes, but it wasn't that bad. I actually work with the layer turned on and a blend mode so that I can clearly see what is going on. This is what the track matte looks like when it is done:

The next step is to repeat the process using a second shape layer with an animated path to cut out the left edge of the picture frame. This is going to require a lot more work, and it could be avoided if your replacement image could have a feathered edge and an edge color that matches the original painting. If you need to do the masking this is what you are facing:

The add blend mode helps me see through the placeholder image and makes it easier to start a new shape layer that can be used as an additional track matte.

Once the track matte layers are good you pre-compose both shape layers and use that as an inverted alpha track matte for the painting. You would then add Camera Lens blur to the track matte so you could match the edges by animating the blur amount.

The last step would be to add your replacement image to the comp, scale it to fit the composition, pre-compose the distorted replacement comp, then use Mocha on the Original footage to apply the corner pin data to the replacement layer. You put the combined track matte above the replacement layer, add camera lens blur to the replacement layer (comp) to match the depth of field changes in the original shot, apply your final color corrections and you are done.  It would probably take me a couple of hours to do this shot correctly. 

After running my quick test of the shot I would probably use this Advanced Corner Pin technique to create the composite because the Roto would be easier to do. There is a lot of copying and pasting in this tutorial that the newer version of Mocha AE makes it easier, but it will probably give you the right idea.

One of these days I'll update that tutorial.

If I was in a hurry I would forget the roto on the picture frame and just put a feathered edge on the replacement image so it would blend in with the original around the edges.

By the way, I did try and run Camera Track on that footage. I could not get enough tracking points to attach anything to the painting or the doorway. Camera Tracking is not a good option for that shot. You could probably get it to work, but it would be a real pain.

[garrobal]

Hi.  Thanks so much for that reply.  That is a tone of info and I will go through it slowly and try it.  I have only been using mocha for 2 weeks and after effects for 2 months so a lot of that will take me time to digest.  I do have one question though.  You mentioned adjusting the surface in mocha.  Did you mean the outer spline?  When I have adjusted the surface it changes/screws up the previously tracked footage.  Unless I am missing something regarding that.  I have come to realize that this is a very complicated piece of footage to work with.  Unfortunetly I spent days figuring out and sucessfully rotoscoping a person that walks in front of one of my painting (not shown in the footage I included) replacements and after days of trial and error I got it perfect and would hate to waste all that work by not being able to accomplish this.  As for the rotoscoping on the right side of the painting I have found that I can simply extend my insert to the right, like the one image you included, and that works ok for this short piece of the video, ingoring that part of the painting frame that would require extensive rotoscoping.  Thanks for all your help! Much appreciated!!  Regards, Gary

[Rick Gerard]

Here's the basic Intro:

Here is some info on Adjust Track. https://lesterbanks.com/2015/11/mochas-adjust-track-feature-can-help-with-drifting-tracks/

This is a direct link to the video on Vimeo, but I strongly recommend that you read the article on the Lester Banks website.

[video]

All of the tracking tools in After Effects require good deal of understanding of the process, now perspective and features change, and practice. There are almost no shots that cannot benefit by a bit of hand tweaking. Your shot is going to require quite a bit.

If the shot was a little better planned and executed the composite would be much easier to do. Planning and executing a shot that is going to work also takes a good understanding of the principals of perspective, paralax, camera position, motion blur, depth of field, and a host of other issues. A poorly planned and executed effects shot can still be successfully completed but the time required to do a good job can grow from an hour or two to a week or two very easily.

It is also very important that you make sure the folks that are presenting tutorials and articles really know what they are talking about. I spend a lot of time on this forum trying to fix workflow problems that folks are having when they try and follow tutorials they found searching the web. A few years ago, when it cost hundreds or even thousanbds of dollars to produce, edit and share screen capture videow only pro's took the time to prepare them, but now, any enthusiast that has a half hour of free time and thinks they have discovered a new magic technique can post something to YouTube. Most of the new tutorials I see promote inefficient workflows and some of them are just plain wrong. Even the fairly good ones leave out a lot of critical info. 

Good luck on your quest to learn AE. Make sure you take advantage of the learning resources that Adobe has vetted and posted on the product page, the User Guide and AE's Home screen. Here's the link: Learn AE. You'll find the link right on the AE page of this forum too.