Make an object stick

hello people, 

i'm making an edit of me standing in front of the camera (in open air) Then I look to my left, and want to show the viewer what I'm looking at. So i turn the camera to the left and a big asteroïd is hanging in the sky. This asteroid is of course a still image.  i want the asteroid coming in with the swipe (and therefore the background)  very smoothly and stay at it's place when the camera movement stops. Tried it uncountable times with position keyframes but it just doesn't work. There's probably a solution here but didn't find it yet. Anyone can give me (a link to) the right answer?