By default only the temporal interpolation is linear. The default spatial interpolation is Auto Bezier; in the YouTube example you show, you can see this in the interpolation lines; they are not straight between keyframes but bend to form curves. Right-click on a position keyframe and open the spatial interpolation sub-menu to verify.
Having said that, I tried to recreate your problem leaving the spatial interpolation at default and moving a piece of media back and forth and was unable to. But I've definitely seen weird behavior like this before, so it's worth checking the spatial interpolation setting. If you've already set it to linear, or setting it to linear doesn't fix the issue, can you recreate the problem in a simple sequence (e.g. moving a solid color around) and upload it? Happy to try it out if so and see if the problem is limited to your system.
As a side note unrelated to your issue, scale keyframes do not have both types of interpolation; if you are doing panning AND zooming and you have both types of keyframes selected, it will only show you the temporal interpolation menu if you right-click.