Position keyframe bounces back from set position

Hi, I have a really simple question which I have spent hours trying to figure out by now. I have a few layers that are masked and are moving in liner X axis. I have tried changin the Key Frame Interpolation to linear and that didn't solve the bouncing problem. 

The keyframes between the time line I'm moving between are set for the same position. That's why I don't get how can the slide move between two keyframes set to same position...

Problem explained in gif below:
https://im6.ezgif.com/tmp/ezgif-6-d4640b96afa1.gif