Sorry, I misconstrued the position keyframes when I said that. It's hard to work out exactrly what's going on with your project without seeing the project, but it's almost certainly an issue with the combined scaling and position change competing with each other, as I mentioned above.
First things to do to diagnose the issue is to remove all spatial interpolation, in case its a bezier curve or temporal issue. Even without eased movements you'll be able to see if the odd jump still occurs. If it does, delete the scale keyframes. Do you still see the jump?
If you're happy to post or message me with your project file I'll work it out in two seconds, but it's a little hard doing it remotely.