Why do my layers move when I go to crop or mask in the animated workspace?

Literally every time I go to crop my gif or mask a group of layers inevitably some layers move on SOME frames or even the mask itself, when all of the layers and frames are in the same placement in a group folder and I am putting the mask on the group, not the individual layers.

I found that if I convert to video timeline and then turn all of my layers into a smart object I can crop the gif without movement but as for masking multiple layers... the only option is to manually re-align the layers in each individual frame.

I am about ready to delete photoshop, I am THAT frustrated.