Help and hints needed for "stabilize" in Tracker

I've got footage that I took via hand held so it is pretty unsteady.  But I have a gray plane (F-22 in this case) against either a nice blue sky or white fluffy clouds.  In both cases, it seems like it would be easy to track.  Many frames have motion blur due to the fact that I can't keep the camera steady but even with the nice clear clean frames, the tracker has problems.  There are times when it can go for 30 to 60 frames which is great but there are also times when it stops after eacy frame and I need to adjust it.

I've set the options so that it stops tracking if confidence is below 80%.  I've tried RGB, Luminance, and Saturation.  I've tried with and without "Enhance Before Match". 

I often set the point on the nose of the plane and set the inner square to about half way up the nose and the bigger square to be about twice as large as the inner square.

Nothing I've tried seems to make a significant difference.  I'm wondering if anyone can share tips.  I think what I don't understand is how the algorithm is trying to find the match because it very often just seems random.