Track Camera + Shape Trouble

If you did not set an origin and ground plane and then set a solid and a camera up using the same points and verify that the origin and ground plane is accurately placed in the shot then you left out one of the most important steps in Camera Tracking. I'm guessing that the points you chose for your target did not line up with the ground plane you needed. There is also a fairly significant valley in the shot so a flat 3D layer is not going to track very well because the perspective changes are not the same for the high and low parts of the shot. You either need a separate plane for each surface or you need to handle the shot entirely in 2D. Personally, I would motion stabilize the shot so the middle of the valley did not move then hand animate a shape layer to fit the changing geometry, then add a null, tie the null's position property to the stabilized footage Anchor Point with a simple expression, then parent the animated shape layer and the footage to the null to put the camera move back in the shot and add the motion to the shape layer. I think you'll have better success than trying to match up a bunch of 3D layers to the multi-plane surface you are trying to outline with your shape layer.