SDK: Mask and checkout layer

Hello,

I've developed a plugin that checkouts a layer and displays it on screen. The plugin applies the same transforms as the original layer (position, scale, rotation... even 3D with OpenGL).

The problem:

When I draw a mask on the solid on which the effect is applied, and I move the mask in the composition, the whole checkout layer moves as well (see images)! Whereas if I apply the mask on the original layer and move it through the comp, the mask will act as "a window", and will only display the section of the layer corresponding to the mask.

- What am I missing here ? 

- Also, why does After Effects display a rectangle as the output and not drawing only the pixels contained within the mask ?

Thank you so much