Swap / replace video with video in after effects: in and out point problems

When I swap / replace video in a comp in after effects, the swapped video has totally different in and out points than I want.

I tried to escape the problem by precomping the video that I want to swap in.
Still after effects manages to shove the footage around and mess up the in and out points. 

Can anyone suggest a remedy?