Skip to main content
K
Khanh32674585851a
Known Participant
October 9, 2023
Question

3D Camera Track and Projection issue

  • October 9, 2023
  • 2 replies
  • 1958 views

So I’ve been trying to recreate this video from Peter Quinn but failed so far. About his video: There is a picture of a guy on a wall, but when you get closer to the wall, the “picture guy” turns into a real person. This is the link of the video: https://www.instagram.com/p/CXahkzhFCWP/


So I’ve followed what he showed in the video above:


- Shoot the end of the video first, freeze frame to make a photo. I divided the original footage into 2 small video: the first one is the guy waving and then walking out of the frame, and the 2nd video is the moving part called BG.
- In After Effects, with the BG video, I reverse it, pre comp then track it using camera track.
- I took Gerad’s advice on the other day: Set the ground plane by using the walkaway, put the grid effect to make sure that the ground layer stays where it should do. Go to the the last frame and select track points on the wall and create a solid layer from here, named it Wall.
- Add the photo that I wanna project it on the wall. Make it 3D, parent it to the Wall solid.
- Add light to project the photo on the wall. Adjust the position of the photo layer in order to align it with the first frame of the video (the guy walking out of the frame).
- Precomp 5 tracks: Light, Still photo, Wall, Ground and 3D tracker camera. In Project panel, duplicate this precomp and change the name into Precomp_Matte. In this new precomp, I put Curves effect into Still layer, make it black and white.
- Back to main comp, drag the Precomp Matte in. In the Precomp, I set Luma Inverted Matte to Precomp Matte to get rid of the white solid (the Wall solid). I precomp 2 tracks again, called it Projected Precomp.
- From here, I check the whole track and you can see it jitters, moving around a bit. I try to come back to the precomp file to adjust the position of the still photo but it’s so hard to see what I’m doing since the white solid gets in the way. Also, my original footage is not great for tracking. I shot it without knowing about the problems can show up later.

 

Any ideas/suggests? Should I use Mocha to fix the Projected Precomp (the one after Luma Inverted Matte) using the BG video? I’m trying to figure out what I’m missing here. Thanks in advance.

This topic has been closed for replies.

2 replies

R
Community Expert
Rick GerardCommunity Expert
Community Expert
October 13, 2023

I took a look at your footage. The first problem is that the footage has a variable frame rate. The first thing you should do before creating a comp or trimming the shot would be to select the footage, go to File/Interpret Footage, and set the frame rate to the standard 23.976 fps.

 

The next thing you should do is trim the footage in the Footage Panel and create a comp with the trimmed footage. 

 

The next step is to split the footage right after the Actor walks out of the frame (Shift + Ctrl/Cmnd + D) and then pre-compose the top copy of the footage, trimming and moving all attributes to the new comp. In the main comp, pick the frame just before the Actor starts waving his hand and use Composition/Save Frame As/Photoshop layers, remove the background from the actor, then save a PNG or PSD file with just the actor.

 

In the first pre-comp, add the PNG file, then Track the trimmed footage in Mocha AE. You'll want to start with a spline that fills the frame (use the rectangle tool) set the tracker to Perspective, about 70%, and start tracking forward. As the footage (at full resolution) is tracked, pause occasionally and adjust the corners of the spline. (Check the AEP file). When you are done, add the Surface tool, expand the surface to fill the frame, (see comp), and then verify that you have a good track by running a preview in Mocha. Save the Mocha Project and return to AE.

 

Extract the Corner Pin data and apply it to the PNG layer. then verify that the track is good. There are some blurred and blended frames and some jerky motion in the footage so adding an adjustment layer above the PNG and adding Pixel Motion Blur will help smooth those things out.

 

The next step is to return to the main comp and Time Reverse the layer using the Layer/Time menu and start lining things up so that you can make a cut to the actor coming to life and walking out of frame. You'll need to apply Corner Pin to the footage of the actor walking out of frame and line up the bricks at the cut point. This is going to give a bit of a jump at the edges when the cut is made, but you can fix that layer. You might need to do a little color correction to match the color of the bricks. You can also overlap the Pre-composed main shot about ten frames and animate opacity to hide the transition.

 

The last step is to Pre-compose again, then scale up Pre-comp 2 a bit in the main comp to hide the edges caused by the corner pin that lined up the shots.  

I hope this helps. 

 

Because of the jitter/motion blur and a couple of other problems, Camera Tracking is not going to give you the kind of accuracy you'll need with this footage. Corner Pin tracking in Mocha AE is almost always an easier solution than camera tracking when you have a good solid surface you want to stick something to.

 

I hope this gets you started. I should add this kind of workflow to the tutorials I am publishing.

K
Khanh32674585851aAuthor
Known Participant
October 14, 2023

Thank you so much for the very detailed instruction. It really gives me hope that I can achieve this even though it might not be perfect as yours. I'm trying to follow step by step right now and will let you know the result.

 

Also, in the meantime, I couldn't open the "Copy 22" file as the error shows up saying "crash occurred while invoking rendering plug in Cinema 4D". I’m on 22.6.0 (Build 64).

R
Community Expert
Rick GerardCommunity Expert
Community Expert
October 14, 2023

Sorry, I made a rookie mistake and had all of the comps set to the C4D rendering engine. The project should have been in Classic 3D.

 

Here's a fixed comp. See if that will open.

R
Community Expert
Rick GerardCommunity Expert
Community Expert
October 9, 2023

The track has to be good. Projecting a light can cause problems if things are not lined up properly, and the projection will only be sharp if a parallel light is used. This way overcomplicates the project. I have already demonstrated and shared the Camera Tracking solution. 

 

I would approach this project using Mocha AE. You will get a better corner pin track using Mocha in less time than you can get with the Camera tracker on the footage you use. I don't have time to create a new tutorial for you, but this is the Mocha/Corner Pin tracking solution that I would use:

 

K
Khanh32674585851aAuthor
Known Participant
October 10, 2023

Thanks for your response.

 

- It’s really hard to verify the track is good or not since my original footage is pretty shaky and has a lot of noise. I did set the ground plane as you said, I watched it when playingback and I thought it was good enough until the jittery effect shows up at the end.


- Yes I do use the parallel light because the projection is blur and it works.


- I used Mocha AE and applied tracking data to the projection comp but it all messed up the scale and position of the precomp. Just to be clear, I do not only want attach the picture to the wall, what I want is to project the picture to the wall using light transmission. I honestly dont know where and when I apply the tracking data from Mocha AE. Like, apply Mocha to the picture before I project it using light transmission, or after it (I already tried this and failed because again, it messed up the scale and position of the picture)? Or to the wall plane since what I need is a good track and good perspective?.

R
Community Expert
Rick GerardCommunity Expert
Community Expert
October 10, 2023

I'm not sure why you are using Projection. The 3D lights and projection workflow will not work if you use Mocha. 

 

If you camera track and you want to use Projection, you will have to establish an origin and ground plane, use the same Target to add a camera and solid, then add a solid to the surface you want to use for the light catcher and make sure that the track is perfect. You then add a parallel light, shift + parent the light to the surface solid, add your artwork (your video), make the layer 3D and shift parent it to the surface solid, then move the parallel light back in Z a bit, move the Projector artwork back in Z, make sure that the point of interest is in the center of the track solid, then release the parenting, set the artwork layer to transmit light, set the shadow catcher on the surface to receive shadows only, then scale the artwork so that it fits the wall.

 

For opaqued artwork, as you have in your sample video, Shift + Parenting, the footage of the man to the Track Solid on the brick wall will give you an easier solution than trying to set up a projector. If the footage does not appear to stick to the wall, then you picked the wrong points to add your solid, or the track is off. 

Firefly Community Gallery
Adobe Firefly

Remix with Firefly Community Gallery

Thousands of free creations to fall in love with and remix in Firefly.

Explore now
Terms & ConditionsAccessibility statement
Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices