Moving Camera + Moving 3D Object

[jerbs]

Hi all

I've tried searching for information, but I may have been using the incorrect terminology.

I have a clip that has a river running from background to foreground. The river starts right-hand background, bends towards the middle of the screen and then exits right-hand foreground. The camera also pans (right to left) during the clip. I'm trying to have a 3D object move down the river (background to foreground). I'm using Element 3D for importing 3D objects.

How do I go about having the 3D object scale/rotate so it moves from the background > foreground, taking into consideration the camera pan. I've found tutorials explaining adding 3D objects (static) to a moving camera and tutorials explaining adding moving 3D objects to a static camera, but nothing that covers both at the same time (moving camera and moving 3D object).

I've managed to create a spline that follows the river - adding a 3D Camera tracking has the spline move with the camera pan. I've attached a 3D object to the spline, but it merely follows the spine/path in a static manner (no rotation around the bend or scale change as it moves towards the foreground).

I'm not sure if After Effects + Elements 3D is capable of accomplishing this, or if I've overlooked information that can explain how to do it. 

Can this be done with After Effects + Elements 3D?

[Mylenium]

Not realyl much to say other than this is a classical "tons of keyframes by hand" job. Pans are lousy to deal with in tracking, anyway, and fixing a botched 3D track can be more painful than just animating stuff directly, even morte so given the limitations of AE's 3D tracker and 3D environment. Could it be done better in another program? Perhaps. but it's still not going to be easy as even in a 3D program you'd likely have to fiddle with scene scale and cheat some depth movement with scaling.

Mylenium

[Kyle Hamrick]

There will certainly be a degree of fine-tuning needed to make this effect work. There may be a few different ways to approach this, depending on the specifics of your shot.

If you're getting a decent-enough track, and your object appears to be following the shot properly, you may try just handling the scaling/rotation manually. 

Element allows you to create a Null for your object groups to follow - I'd highly recommend using this feature, and trying to match the appropriate movement using this. If you're getting a proper track, you ideally shouldn't need to worry about scaling, as proper z-depth movement should handle the visual size change of the object. Since you're tracking a 3d object over 2d footage, there will likely be some trial and error to find the right positioning that actually looks correct and follows the shot properly.

If you'd like to post the shot itself, we may be able to give better, more specific advice.

[Rick Gerard]

Sometimes you can combine Mocha AE's corner pin tracking applied to a solid with a grid to get Camera Tracking to work on a shot. I do it all the time to convert AE's camera tracking to 3D tracking when I have moving objects in a scene I need to follow. 

If you can post the video I will be able to tell if that technique will work. If it will not your only option is to create a small 3D solid, add the grid effect, estimate the focal length of the camera used to take the shot, then rotate the 3d solid -90 in X and position it in the scene by adjusting the X Y and Z position so that it looks like it is matching the plane of the river. The grid will help you figure out where the layer needs to be. 

Here's the exact workflow that I follow to match the perspective and camera angle in a shot. 

1. If the footage is video with a panning and tilting camera, pick a frame somewhere in the middle of the shot

2. Add a camera using your best guess for focal length, open up the point of interest and set a keyframe at the comp center to easily lock it there or add this expression to make sure it does not move

[thisComp.width/2, thisComp.height/2, 0]

3. Reveal the Zoom and Position properties in the Camera Layer and add a keyframe to Zoom, add a keyframe for position, then add this expression to the camera position:

[thisComp.width/2, value[1], - cameraOption.zoom]

The expression will only allow you to edit the Y position of the camera

4. Add a solid to the timeline the same size as the comp, add the grid effect, make the layer 3D and rotate the layer -90 degrees in X (most of the time I just add the expression -90 to the X rotation)

5. Press U to reveal the zoom and position property of the camera and adjust only the Y position of the camera and move it up until the grid aligns with the perspective in the shot. Fine-tune the camera position by adjusting the zoom value which automatically adjusts the position and returning to the Y position until you are pretty sure the position matches. The Y value adjusts the camera angle, changing the zoom value changes the camera position

If I was trying to get add a boat to the river I would create another solid to represent the boat that was about 1/4 the size of the comp and probably add a grid. By far the easiest way to do that is to duplicate the guide grid, open up the layer properties for the duplicate, change the size, and maybe the color and select, and deselect All Layers so you get a new layer that is all ready to go.

Because the river is lower than the reference grid you want to keep things lined up you can either parent the Boat layer to the Grid layer and use the Timeline to scrub through values to animate the position, or you can make sure that the selection tool is set to Local Axis and move the layer around carefully using only a single axis at a time in the Comp panel. That is the method that I prefer. Moving the boat down and adjusting the left and right it is pretty easy to get the boat at the right depth and in the right position. With just a few keyframes and Two views enabled so you can adjust the motion path Bezier handles with the pen tool (g), you can successfully draw and position the boat so it moves down the river.

After Verifying that the motion path follows a flat plane by checking the front or side view the only remaining task is to add some rotation keyframes. 

There you go. A boat placeholder moving accurately in 3D space in about 10 minutes. If the shot has a moving camera you have to jump through a few more hoops, but it is still a fairly quick manual task.

Why does this work? No matter where you point a camera the point of interest is in the center of the frame. Perspective is controlled by camera position and framing is controlled by the focal length. This technique allows you to easily experiment with camera height and distance which is all you need if the camera is level. 

This workflow is part of a new tutorial series and article that I am about to publish. I'll post on the forum when it is ready.

I hope this helps, but once again, if you show us the shot we can point you to the most efficient solution.

In case you are interested, this is the image from Adobe Stock, and this is the comp (2020).

[jerbs]

Thanks for the quick and concise replies!

I had a feeling that frame-by-frames adjustements would be required. Video footage can be seen here. But I've also included a frame (below), which provides an idea of how rivers shape. Rick_Gerard, it looks similar to the example you provided.

Rick_Gerard, I really appreciate the method you provided. I'll use it and play around on a static background image for practise (can never get enough practise!).

Based on your experience and now-knowledge of the video, what would be the ideal option to take?

[Rick Gerard]

I downloaded the footage. With a few enhancements, the shot should be able to be camera tracked. If it can I'll let you know. I just started the camera track. If it is successful I will report back with a workflow.

[Rick Gerard · Accepted solution]

I spent most of tonight working on another project. When I needed a break I jumped into your project. I spent about 3 minutes on the project at about 9:35 setting up the Camera tracking. Here's the workflow.

The first thing I did was trim the footage, create a new comp from the footage, Duplicate the footage layer, then pre-compose it and add these effects to the footage to generate as much detail as I could get from the footage. Here are the details:

You can see everything I did in the timeline. It's ugly, but there is a lot of detail for the tracker to look at, especially in the distance.

Then at about 11:30 I picked a good spot with a lot of detail, set a marker, picked and set the origin and ground plane, then added a Camera and a solid. The Solid was resized to make it big enough to cover the river and I added a mask for the river and a smaller mask where a boat was in the middle of the river. That process took about another 5 minutes. Just before I stopped I added a 3D null, set the position of the null to 0, 0, 0  because that is the origin, then parented the Camera and the Solid to the null and reset the position of all the 3D elements to comp center. Then I locked the layers. 

At about 12:15 I went back to the project and added a third mask path to the Origin solid using 2 views so I could get the boat exactly where I wanted it. I added a small green solid to the comp to represent the boat, made the layer 3D, which snapped it to comp center and the origin of the camera tracker. I then set a single mask keyframe for the motion path mask on the Origin solid, cut it, then set a Position keyframe for the boat solid and hit paste. This gave me the perfect motion path but it was vertical so I added a Rotater null, made it 3D parented it to the Boat Null. and rotated it -90º in X to line up the motion path with the Origin solid. 

A few keyframes and 10 minutes later I had a green solid sailing down the river. After Removing all of the parenting I exported the Comp as a Maxon C4D file which could be used to add my 3D boat to the scene. All of the layers, the camera movement, and the animation came over perfectly. Maybe sometime tomorrow I'll add a boat. Buy the way, You can also add lights and images you can use for texture and environment maps to the AE project and they will all come into C4D so you can build some great textures for your boat and Make it look real. You could even generate a wake in AE to make it look even more real. 

Here's a sped-up screen cap movie of the process. You can see the clock ticking. 

[jerbs]

That is a lot to digest, but thank you for taking the time to show how it can be done. I'll work through and see how I go. Thanks, Rick!