I cannto deteremine from your screenshoo what is your compound mask shape. I like to fill my mask with a dummy color prior using the mask command. The color disappears after executing the mask, so does not matter what it is. This helps you confirm the mask is on top, and the lemensst yo wish to mask are underneath. Use the solid arrow tool so you select entire group/compound.mask, etc.
Try with a simple rectangle as your mask shape. If it works, undo. Yo then know the issue si with you mask shape and not the items you are trygn to mask. You many need to break the compound shape for the mask, and recreate,
You can also use the transparency palette >> Make Mask to create mask. Black wil drop out to transparent, and white will keep the art at 100%. Group the items you wish to mask together as your art, then on top have a mask shape.