Hello. To find a 2D layer's efficient bounds, we checkout the layer with the request rect:
PF_RenderRequest req = extra->input->output_request;
ERR(extra->cb->checkout_layer(in_data->effect_ref,
SKELETON_INPUT,
SKELETON_INPUT,
&req,
in_data->current_time,
in_data->time_step,
in_data->time_scale,
&in_result));
The in_result.result_rect is the efficient extents of the vector layer. Turn the layer to 3D, the extra->input->output_request is the same size (the composition dims) except checking out the layer now gives an in_result.result_rect as the dimensions of the entire composition, which is not efficient at all especially at high resolutions. If the layer is 10px by 10px in a 4k composition, the input is now 4k.
Is there a way to get the efficient rect? I tried AEGP_GetLayerMaskedBounds which returns the efficient floating point bounds in layer space. I then tried to get that in world space by plugging it into a matrix4 and multiplying it by the layer2World matrix returned from AEGP_GetLayerToWorldXform but I must be doing something wrong because it's not working correctly. I figure this workaround is overkill and there's likely something I'm missing, if someone could please point me in the right direction. Thank you.