Tracking scaled down/resized video layer

Here is a project that I am working on. I'll start with the design goal. We have a shot of a theater that could only be shot in the daytime. The script calls for the shot to be late evening. The theater marquis needs to be lit up. Dancers need to be silhouetted in the top windows on the west (left wall), the posters need to be illuminated. Lights need to be on in the lobby. The lights in the car that passes need to be on. All of the hanging lights need to be on. and the lighting needs to be cleaned up. This is what my setup in Mocha looks like so far:

The first thing I did, the top layer in Mocha, was to track the foreground flag, then lock the layer, then create 2 masks for the flag and the pole and lock them to the Foreground Track layer. It required a couple of keyframes for the masks to be tight enough. That layer was not tracked but locked to the track and named Foreground Matte (2).

The next step (layer 3) was to track the left wall of the building. I used a couple of splines and perspective tracked the wall, then locked it and turned off tracking. Then I created a pair of masks for the right windows on the same layer (4) Right Windows because I have a shot of two dancers that will work for those two windows, Because of the parallax change for the inset windows, after locking the window masks to the wall track I needed a few keyframes, but it only took a minute to setup. I did the same for the next window because the other shot I have is just for one dancer. Then I added another layer for the right poster and adjusted the surface to cover the poster. There is a grid on that shot. The Left and right window layers and the right poster are locked to the wall track because they are all on the same surface. No tracking was necessary. 

Then I created s spline for the inside wall and a surface for the second poster. That layer was tracked and locked. Then I tracked the Marquis and added a surface to act as the lighted sign.

Back in After Effects, I started building the comp by creating the matte layers for the foreground and the Right Wall. 

I added my footage for the pair of windows on the right, pre-composed, opened the pre-comp, added a viewer, split the timeline so I could work on both layers, and set up the composite. This screenshot shows the composite with a few of the elements turned on. A scratch Day for Night Lut is applied to the bottom layer (the original shot). One of the streetlights has been added and it follows the streetlight by simple motion tracking. The footage has been added to the windows, and the lighted sign and posters have been added. You can see how I scaled the footage of the dancers to fit the window. The Cancers comp was corner pined using the two window surface I set up in Mocha, I added a few effects. and the composite looks pretty good for this point in the process.

The thing that makes the composite work is the Matte I created for the foreground so I'll explain that. I took a copy of the original footage and added curves to get a very high contrast version of the footage that I could lay on top of the footage to preserve not only the window frames but the light bulbs and wire for the hanging lights. That composite looks like this:

There is still a long way to go. I have to create another matte for the window to the left of the solo dancer. I need to track the lobby in Mocha and add some masking and a surface there. I need to track the cars that move through the shot. I need to track the strand of lights in the foreground and add lighting effects to all of them. To complete the shot I'll need a half dozen more tracks in Mocha, maybe a Camera Track, and a bunch more layers and masks. This is the same workflow you would go through Davinci Resolve, Nuke, Fusion, or any other compositing app. Track the large surfaces, lock mattes to the surfaces, lock surfaces to the surfaces you corner pin track, add layers. The process is the same, only the tools are different.

One thing that I forgot to mention. The original footage is 4K, the comp is HD, the original footage was scaled and moved to fit the framing we needed, and the pre-composed original footage layer is what was used for all of the motion tracking. Here is a test render of the comp at this point.

Do you see the error I have to fix? It's the foreground matte. I didn't track it far enough in reverse so it does not accurately mask the streetlight's glow. That's an easy fix. Just open Mocha again and run the track in reverse, then export the matte again. It will take about 5 minutes.

This is as close as I have to a project like yours. It's a lot more complicated. From my guess at what the footage looks like you would need two or three corner pin tracks of TVs on the same plane, then a bunch of surfaces and masks that could be on the same layer just linked to those tracks. There is no reason at all you couldn't do everything in Mocha. 

If this shot was for a feature film I would be putting reflections in the building windows on the right, probably add people on the sidewalk, a neon sign to the small shop to the right of the theater, and a bunch of other things. I can identify about 15 surfaces that can be tracked in the original footage. If you track them one at a time, then lock them, you can add as many surfaces as you like or create as many mattes as you would like without adding any workload or additional rendering with mocha. I have had projects in the past with over 100 mattes in a single shot. 

I hope this helps. When I get my Mocha AE tracking tutorial series completed I'll post a link.

I assume Layer 1is the shot with all the Television sets stacked up that you are corner pin tracking with Mocha AE. You have added Warp Stabilizer to the shot so you should either take it off and motion track or pre-compose the footage moving all attributes to the new comp before you do the corner pin tracking. Warp stabilizer is going to throw off the 
Corner Pin tracking a little bit, so either get rid of it or pre-compose.

I assume that Layer 2 is the replacement screen, but its scale is 100%. It just looks like a movie, not a movie of something happening on a TV screen, but a movie of somebody in a room. If you apply the corner pin track to Layer 2 it will line up with the surface you setup in Mocha AE. If the surface needs to be smaller, make it smaller. If it needs to be bigger, make it bigger.

If you need to scale Layer 2 so that it is smaller, then you either have to use the Stabilized corner Pin workflow I outlined or you are going to have to scale it down and pre-compose it before applying Corner Pin. The layer will match the size of the surface you defined in Mocha AE.

If you need to motion stabilize Layer 2 and scale it, the same thing applies. Do all that work, then pre-compose, then apply the Mocha AE corner pin to the pre-comp.

If you open the project I prepared, opened it up, and replaced either of the two bottom layers with a movie file, made the movie fit the comp, and then pre-composed it if you had to scale it, the comp would work perfectly. 

Your cropped screenshots are a little better but they still don't tell the whole picture. I don't know what layer 2 looks like. I'm guessing that it is some kind of moving camera shot on that stack of televisions. I don't know what Layer 2 looks like either. From what I can tell it's just a shot of somebody sitting in a chair. I don't know why you need to scale the movie. If it fits the comp frame, then it will fit the surface you define in Mocha. If it does not fit the screen, make it fit, then pre-compose and it will fit on the surface you defined in Mocha. 

If the shots are as I described, then it should take you just a few minutes to track it in Mocha and replace one of the screens. If the master shot of the TV screens has so much camera shake that it needs to be warp stabilized, then the very best option is to do all of the tracking and screen replacement before you warp stabilize, then pre-compose the entire composite and run Warp Stabilizer on the entire shot. That way the warping will be the same on the entire shot, instead of having the screen replacement not warping and the main shot warping to take out the camera moves. 

If you can share just a few seconds of both clips I'll run the composite for you. If you can give us a detailed workflow from start to finish I can probably help you find the user error. 

I've gone over this with you before. Is it the same shot? 

Mocha AE will accurately track almost any surface and you can generate corner pin data to fit any comp-sized layer inside that surface. 

If the video you want to resize is not the same size as the comp then all you have to do is to go to the Layer/Transform/Fit To Comp menu to fit the contents of the layer to the comp. Then pre-compose moving all attributes to the new composition. Now the layer is ready to receive the corner pin data and it will snap perfectly to the surface you tracked.

Leave out adjusting the surface in Mocha and this won't work. It is very difficult to find a shot that has a screen in it that Mocha cannot track and successfully replace the screen. 

If the layer you need to track has been resized, is not the same size as the comp, or has any other effects or transforms applied and you want to track it, pre-compose it first, moving all attributes to the new comp. It's as simple as that.

Show us your comp with the shot in it and all the modified properties of the layers. Show us the shot you are tracking in Mocha with the surface turned on and a placeholder like a grid in the shot.

Try downloading the comp that I posted and opening it if you are lost. Try looking at a few of the Mocha AE tutorials you can find from the Help menu in Mocha. You should take a look at this one. It talks about when layer size does not match the comp size.