How toComp(value) works with Shape layers?

A parametric shape like a rectangle has a Rectangle Path position that defaults to 0,0 when you draw a new rectangle. The rectangle also has a Transform Rectangle Anchor Point (default 0, 0) and Transform Rectangle Position, as well as Skew, Skew Axis, Rotation, and Opacity. The Transform Rectangle Position is based on the layer center, not the Layer Position or Anchor Point. The default is 0, 0.

The shapes' actual position on the screen is a combination of all of those factors, so if you want toComp or toWorld to have any relationship to Rectangle 1, you have to factor in Contents/Rectangle 1/Rectangle 1 Path/position and Contents/Rectangle 1/Transform Rectangle 1/Position and Anchor Point.  If the shape is a path, there is no Shape/Position property to deal with, so the calculations are a little easier to come up with.

Exactly what are you trying to accomplish? The expression language toComp and toWorld operators are intended to return a layer's coordinates when nested. You'll have to throw in the shape layer coordinates to figure out where they should be, and I'm not sure what toComp or toWorld would have to do with that unless the layer was a Child of another layer or you needed 2D values to relate to the 3D position of another layer.  

If you want to fix something to the entire contents of a shape layer, you can use sourceRectAtTime(). You can easily find the center, corners, or center of the edges with a simple expression.

ref = sourceRectAtTime(); // your source layer
w = ref.width;
h = ref.height;
l = ref.left;
t = ref.top;

center = [l + w / 2, t + h / 2];
topLeft = [l, t];
topMiddle = [l + w / 2, t];
topRight = [l + w, t];
rightMiddle = [l + w, t + h / 2];
bottomRight = [l + w, t + h];
bottomMiddle = [l + w / 2, t + h];
bottomLeft = [l, t + h];
leftMiddle = [l, t + h / 2];

topLeft

This works for any layer and can be modified to factor in layer scale, toWorld, toComp, and rotation. You cannot compensate for masks, but maybe something like this will get you where you need to go. 

That expression, along with an Effect/Expression Controls/Drop Down Menu Control, is one of my top 10 Animation presets for Anchor Point.

One more thing. A Shape Layer size is always the same as the comp size, and the Anchor Point is always at 0, 0 by default unless the Shape Layer was created from a Vector Layer (AI file). In that case, the Shape Layer size is the same size as the Vector Layer used to create the Shape Layer and the Anchor Point is in the middle of that layer. If you are converting Illustrator layers to Shape layers to create masks or motion paths, you have to compensate for the Anchor Point and position properties of the original Illustrator layer. 

Shape layers use a centrered coordinate system and thus the actual values will always be offset by half the comp height and width. You manually have to account for that in your expressions code. I agree that the behavior is inconsistent due to this design decision, but that's just how it is. Using continuous rasterization will similarly require a different approach using toWorld() and fromWorld(). Everything perfectly doable, there are just a few more layers of code to this than the typical tutorials on YouTube may make people think.

Mylenium