Image tracking - how to stabilize object?

There are two reasons that Camera tracking does not work correctly. The first is lens distortion. Wide-angle lenses distort near the edges; you must correct for lens distortion before the tracking has any chance of working at the edges of the frame. The second reason the tracking fails is that you did not select tracking markers that were all on the same plane. You must be very careful, especially in wide-angle drone shots, to select co-plainer markers accurately, set an origin and ground plane, and then verify that the added solid sticks to the surface. 

I suspect the tracking error was caused because you did not select tracking points on the same plane. This is a common mistake when Camera Tracking drone shots with wide-angle lenses and tall trees or buildings in the shot. 

I would select Detailed analysis in Camera Tracker, and then I would hold down the shift key and drag selections around small areas on the ground, careful not to include any trees. If you do that and include foreground and background areas, you should get an excellent average of the ground plane. You can then drag the target to about the middle of the shot, right-click, set an origin and ground plane, then use the same target to create a solid and a camera. I always make the solid a guide layer so it won't render. Then I create a new shape layer, make it 3D, and Shift + parent it to the Ground Plane layer. If you hold down the Shift key while parenting, the child layer will snap to the same position and orientation as the Ground Plane layer. You can draw some shapes on landmarks and check the track.

When the track has been verified, the last step is to lock the ground plane guide layer and add the other elements to the scene. You can use a single tracking point or select multiple points on the same surface, then move them around as needed to set up your composite.

I took your shot and set up the tracking so that you can get accurate results. Here's the quick tutorial. I hope it is OK that I used a copy of your failed track as the source. 

More tracking tutorials are on the way.

My Quick Tips Playlist

–

Without any info about your settings and the clean plate nobody can tell you much, but the answer you are not going to like is that the shot likely is not suitable for 3D tracking, at least in AE, if underneath that solid there aren't a ton of markers or distinct features like other buildings, electricity poles, roads or whatever. 3D tracking is not a magic procedure, it still requires to plan the shot properly and pick a suitable methodology.

Mylenium