Mask Tracking Manually With a Small Mask

I'm not sure if this is a bug or just user error, but man it's been a frustrating morning. I am trying to blur a license plate on a car that is backing into a parking space. The auto mask tracking will not work because the car swings in slowly from an angle, so the plate changes both size and direction. I've tried separating into multiple clips to see if that helps to no avail, so I assume it's down to frame-by-frame tracking.

The problem seems to be when I create the original mask, it is considerably small, so when I try to make any slight movements, the hand will not appear. It only allows me to adjust the frame points with the pen, not move the mask itself for smooth tracking. And there doesn't seem to be XY controls either. I've tried auto tracking in reverse as well, which works a little better but still ends up looking pretty sloppy.

Any insights? Other methods I should try? It's such a seemingly insignificant part of the video, I hate that it's eating up so much time and energy. Plus, I need to recreate this same process with five other videos, so I'm losing hair by the second haha.

TIA

-Luke