Adjustment layer problem

The problem I've been facing is when my adjustment layer has effects added to it that makes a distortion or something under footage or effects, it sets it in the middle of the screen and the effect doesn't follow the position of the footage or effect.

Some examples of the effects are:

Displacement map

Chromatic displacement

Compound blur

and more...

This is very important because I am a VFX editor and I need this to work for my videos.