Time Remapping Moves my Keyframe in the timeline.

Hi, Im not sure if i am just misusing the feature or if my settings are weird. I have watched some videos and tried to figure this out but my premiere seems to be actin differently. 
When i use the speedramp function in the TL with the rubber band. I add a keyframe using the pentool, say at 5 seconds, then to the right of the keyframe I just placed I try to shift drag upward to speed up or slow down the footage. When i do this my keyframe moves to a different spot, say 7 seconds. But i need the ramp to start at 5? I dont understand why it keeps shifting. I get why the end of the clip would shorten or lengthen but why wont the keyframe stay where it needs to?? I watched a video where a guy did litterally the same thing and his key frames stayed right where they were, no problem 
I hope these photos demonstrate what i mean.