Get LayerMask histogram by layer ID

[Marvin01]

I have this working action manager function that identifies an empty mask (all black). But it uses the currently selected layer as a reference. I would like to change this function to work with an layer ID, without selecting the layer, but I can't seem to get it to work.

Working script:

function actionManagerUserMaskEmpty(){
    
    var maskEmpty = false;
    var totalPixels = app.activeDocument.width * app.activeDocument.height;
    
    var r1 = new ActionReference();
        
    r1.putEnumerated(charIDToTypeID('Chnl'), charIDToTypeID('Chnl'), charIDToTypeID('Msk '));
        
    var layerMaskProperties = executeActionGet(r1);
    var histogramArray = layerMaskProperties.getList(charIDToTypeID('Hstg'));
    var blackValue = histogramArray.getInteger(0);
        
    if (blackValue === totalPixels) {
        maskEmpty = true;
    }

    return maskEmpty;
}

alert(actionManagerUserMaskEmpty())

[r-bin · Accepted solution]

Your script works correctly if the ruler is set to pixels.

Try this option. Here 5 is the id of the layer.

function actionManagerUserMaskEmpty(id)
    {
    try {
        var r = new ActionReference();    
        r.putProperty(stringIDToTypeID("property"), stringIDToTypeID("hasUserMask"));
        r.putIdentifier(stringIDToTypeID("layer"), id);
    
        if (executeActionGet(r).getBoolean(stringIDToTypeID("hasUserMask")) == false)
            {
            return -1; // no mask
            }

        var maskEmpty = true;

        var d = new ActionDescriptor();
        var r = new ActionReference();
        r.putProperty(stringIDToTypeID("channel"), stringIDToTypeID("selection"));
        d.putReference(stringIDToTypeID("null"), r);
        var r1 = new ActionReference();
        r1.putEnumerated(stringIDToTypeID("channel"), stringIDToTypeID("channel"), stringIDToTypeID("mask"));
        r1.putIdentifier(stringIDToTypeID("layer"), id);
        d.putReference(stringIDToTypeID("to"), r1);
        executeAction(stringIDToTypeID("set"), d, DialogModes.NO);


        var r = new ActionReference();    
        r.putProperty(stringIDToTypeID("property"), stringIDToTypeID("selection"));
        r.putEnumerated(stringIDToTypeID("document"), stringIDToTypeID("ordinal"), stringIDToTypeID("targetEnum"));

        if (executeActionGet(r).hasKey(stringIDToTypeID("selection")))
            {
            maskEmpty = false;
            app.activeDocument.selection.deselect();
            }

        return maskEmpty;
        } 
    catch (e) { alert(e.line+ "\n\n" +e); }
    }

alert(actionManagerUserMaskEmpty(5));

[Marvin01]

Thanks for you working solution. My solution takes about 13 seconds for 90 layers while yours executed in 11 seconds. I would still love mine solution to work without selecting the layer because I can also detect fully white masks, something I need for another process. This is easily done with the histogram data. Are you sure my function annot be modified to target the layer directly without activating it first?

[r-bin]

You are working with a histogram of channels. You can make a layer mask channel active (to access it) only by making the layer active. I don’t know any other way and it’s unlikely that there is one if we talk about scripts.

To determine that the mask is completely white, do the following in code. After determining that the mask is not black, invert selection. If selection disappears, then the mask is all white.

You can also convert the selection to a Quick Mask and then you can access the Quick Mask histogram, which in turn is a copy of the selection, which is a copy of the layer mask channel.

var r = new ActionReference();
r.putName(stringIDToTypeID("channel"), "Quick Mask");
var hst = executeActionGet(r).getList(stringIDToTypeID("histogram"));