How to get the visible bounds of a layer set (group)?

[adenry]

Hello world!

I'm writing a simple script where I need the bounds of the visible portion of the active layer (i.e. the bounds after the mask is applied).

Thankfully, the ArtLayer.bounds property gives you exactly that for any given layer.

However, if the active layer is actually a group (layer set), things get complicated.

First of all, the group's mask effect is completely ignored.

And second, the visibility state of the layers is being ignored as well.

So my question is: how do I get the bounds of the visible portion of a layer set (group)?

Or at the very least, how do I get the bounds of the group's mask? If I have that, I can iterate through all the group's visible layers bounds and intersect all of them with the mask bounds to get what I need.

I didn't find any methods or properties in the reference guide that could give me either of those.

And one bonus question: how can I find out if the active layer is an ArtLayer or LayerSet?

Any help would be appreciated!

Thanks!

[nvkzNemo]

1. Merge group

2. Get bounds

3. Undo (Step backward in history)

[SuperMerlin]

alert(activeDocument.activeLayer.typename);

[adenry]

SuperMerlin‌ thank you, that answers the bonus question (duh, silly me...).

nvkzNemo‌ thank you, that's a working solution, I tested it, it does the job. However, I would still like an alternative solution that doesn't require such a potentially heavy operation like the group merging. Any idea how I can get a LayerSet's mask bounds? Even though my idea of bounds intersection would require more complicated coding, it should ultimately be much faster.

[nvkzNemo]

This will may be working, but only if group has mask (vector or raster)

var ref = new ActionReference();
ref.putEnumerated(charIDToTypeID('Lyr '), charIDToTypeID('Ordn'), charIDToTypeID('Trgt'));
var bounds = executeActionGet(ref).getObjectValue(stringIDToTypeID("bounds"));
var top = bounds.getDouble(stringIDToTypeID("top"));
var left = bounds.getDouble(stringIDToTypeID("left"));
var right = bounds.getDouble(stringIDToTypeID("right"));
var bottom = bounds.getDouble(stringIDToTypeID("bottom"));


alert("Top: " + top + "\nLeft: " + left + "\nRight: " + right + "\nBottom: " + bottom);

I've checked in the CC 2015

[adenry]

nvkzNemo Fantastic! This works like a charm, the bounds are always correct for any layer or group, whether they have masks or not!

Just one final question, if I may. This code works on the active layer, I suppose because of this part:

charIDToTypeID('Trgt'));  

Since I need to iterate through all the group's layers and subgroups, is there a way to get a unique ID for every child layer/group, and run this bounds code for that ID instead of for the active layer (or something like that)? How would I do that?

I could change the document.activeLayer for every layer in the LayerSet, but I would like to avoid that, if possible.

In short, is there a way to address a layer/group by a unique ID?

Thanks!

[matias.kiviniemi]

Replacing the putEnumerated-line with below should work (at least in the cases where I use it)

ref.putIdentifier(charIDToTypeID("Lyr "), layer.id)

[nvkzNemo]

This function will return the active layers ID

function getLayerID() {
    var ref = new ActionReference();
    ref.putEnumerated(charIDToTypeID('Lyr '), charIDToTypeID('Ordn'), charIDToTypeID('Trgt'));
    return executeActionGet(ref).getInteger(stringIDToTypeID("layerID"));
}

This function will return the layer bounds by ID

function getLayerBoundsByID(_id) {
    var myObj = {};
    var ref = new ActionReference();
    ref.putIdentifier(charIDToTypeID('Lyr '), _id);
    var bounds = executeActionGet(ref).getObjectValue(stringIDToTypeID("bounds"));
    myObj.top = bounds.getDouble(stringIDToTypeID("top"));
    myObj.left = bounds.getDouble(stringIDToTypeID("left"));
    myObj.right = bounds.getDouble(stringIDToTypeID("right"));
    myObj.bottom = bounds.getDouble(stringIDToTypeID("bottom"));
    return myObj
}

[adenry]

nvkzNemo‌, matias.kiviniemi‌, thank you for your help!

I'm almost done with my script, however I realized I actually need to apply all masks to the layers before getting their bounds, since simple intersection of the masks bounds with the layer bounds doesn't give accurate results when there is more complicated layer content.

I managed to come up with functions to rasterize the vector mask and to apply the raster mask.

However, first I need to check if the masks aren't disabled (by shift-clicking them in Photoshop, bringing a big red "X" on them).

So I came up with these functions:

function vectorMaskEnabled( _layerID ){  
     var ref = new ActionReference();            
     ref.putIdentifier(charIDToTypeID('Lyr '), _layerID);  
     return executeActionGet( ref ).getBoolean( stringIDToTypeID( 'vectorMaskEnabled' ) );  
}

function rasterMaskEnabled( _layerID ){  
     var ref = new ActionReference();  
     ref.putIdentifier(charIDToTypeID('Lyr '), _layerID);  
     return executeActionGet( ref ).getBoolean( stringIDToTypeID( 'userMaskEnabled' ) );  
}  

For whatever reason, the rasterMaskEnabled function works, but the vectorMaskEnabled does not. It seems like "vectorMaskEnabled" is always true when there is a vector mask, regardless of its state (enabled or disabled), where "userMaskEnabled" works as expected.

Any idea why is that happening and how can I fix it? I just need to know if the vector mask is enabled or not.

I have to say, scripting for Photoshop is really not a pleasant experience. All that action manager stuff with the ScriptListener plugin is very annoying.