3D After Effects layers and rotation

I am trying to rotate four extruded 3D layers at the same time. The idea is that it's a medal or medallion. However, the layers are moving, from what I can tell, based on the individual anchor points or layer center location. This gives certain elements a bobbing up and down effect, or pieces of each layer blending through the back and side of the medallion.
I needed the four layers extruded at different heights, that’s why I have them on different layers. Is there a way to group them to stop unnecessary movements? Or am I’m missing what I need to align the layers to? I appreciate any advice.