Rounded Rectangle Mask via Scripting

[LiamCarlin]

Hey Guys,

I am creating a script that applies a round rectangle mask to any layer that is selected. I have written the following code.

My question is, how do I make the vertices scale correctly to the size of any given comp? For example the code below has the vertices hard coded for a compostion of 1920x1080. How can I make this dynamic so it would work the same on an 8k comp for example.

Thanks,

Liam

btnMaskRoudRect.onClick = function() {
    if(app.project.activeItem == undefined || app.project.activeItem == null) {
        alert("Please Select a Composition.");
        return false;
     }
     if(app.project.activeItem.selectedLayers.length < 1) {
         alert("Please Select a Layer.");
     }
 else {
    maskRoudRect(app.project.activeItem.selectedLayers);
     }
     function maskRoudRect() {
        for(var i = 0; i < app.project.activeItem.selectedLayers.length; i++) {
        app.beginUndoGroup("Undo Rounded Rectangle Mask");
        myLayer = app.project.activeItem.selectedLayers[i];
        comp = app.project.activeItem;
        newMask = myLayer.Masks.addProperty("ADBE Mask Atom");
        newMask.maskMode = MaskMode.ADD;
        myProperty = newMask.property("ADBE Mask Shape");
        myShape = myProperty.value;
        myShape.vertices = [[1900, 0],[20, 0],[0, 20],[0, 1060],[20, 1080],[1900, 1080],[1920, 1060],[1920, 20]];
        myShape.inTangents = [[11.045654296875,0],[0,0],[0,-11.457153320312],[0,0],[-11.457153320312,0],[0,0],[0,11.045654296875],[0,0]];
        myShape.outTangents = [[0,0],[-11.457153320312,0],[0,0],[0,11.045654296875],[0,0],[11.045654296875,0],[0,0],[0,-11.457153320312]];
        myShape.closed = true;
        myProperty.setValue(myShape);
        app.endUndoGroup();
    } 
}

[Mylenium]

Obviously you'd have to retrieve the comp size and insert the values as variables, none of which exists in your current code.

Mylenium

[LiamCarlin]

Thanks for replying Mylenium.

Correct, they don't. I can retrieve those values via comp.width and comp.height.

My issue is, once I have these, how do I use this to calculate where the vertices should be in any given comp?

Thanks,

Liam

[Mathias Moehl · Accepted solution]

say a vertex is at [x,y] in a comp with 1920x1080 and you want to scale everything to fit your comp.

Then you can do

var scaleX = comp.width/1920;
var scaleY = comp.height/1080;

scaledVertex = [x*scaleX, y*scaleY]

[LiamCarlin]

Thanks Mathias and Mylenium, this works well for scaling the vertices.

One thing I have noticed is when the vertices are scaled, the bezier handle do not adjust accordingling. Please see attached image. The 1080p image works perfect, the 4k image is lacking that roundness.

The value of the in/out tangents doesn't change but the bezier handle does. Why is that? Is that in relation to the bezier forumla or Is this a different property all together?

Thanks,

Liam

[Mathias Moehl]

You need to scale the inTangents and outTangents, too (using exactly the same formula).
The handles in your 4K comp have exactly the same length as the ones in the HD comp. But since the shape itself is much bigger in the 4K comp, the handles are smaller relatize to the size of the shape itself.

[LiamCarlin]

Perfect! Much appreciated Mathias and working well.

Thanks,

Liam 

[Mathias Moehl]

You are welcome 🙂

[LiamCarlin]

Apologies Mathias, I do have an additional question if I may.

The codes works as intended however I hadn't anticipated for layers to be of a different size to the compistion. The images below show a solid layer that is 5000x5000 with out a mask and then the mask applied (please see code below). How would I compensate for the size of the layer? Would I need to use sourceRectAtTime to find out it's size as opposed to using the comp.widthscaleX, comp.height*scaleY.

btnMaskRoudRect.onClick = function() {
    if(app.project.activeItem == undefined || app.project.activeItem == null) {
        alert("Please Select a Composition");
        return false;
     }
     if(app.project.activeItem.selectedLayers.length < 1) {
         alert("Please Select a Layer");
     }
 else {
    maskRoudRect(app.project.activeItem.selectedLayers);
     }
     function maskRoudRect() {
        for(var i = 0; i < app.project.activeItem.selectedLayers.length; i++) {
        app.beginUndoGroup("Undo Rounded Rectangle Mask");
        myLayer = app.project.activeItem.selectedLayers[i];
        comp = app.project.activeItem;
        newMask = myLayer.Masks.addProperty("ADBE Mask Atom");
        newMask.name = "Rounded Rectangle Mask";
        newMask.maskMode = MaskMode.ADD;
        myProperty = newMask.property("ADBE Mask Shape");
        myShape = myProperty.value;
        var scaleX = comp.width/1920;
        var scaleY = comp.height/1080;
        myShape.vertices = [[1900*scaleX, 0*scaleY],[20*scaleX, 0*scaleY],[0*scaleX, 20*scaleY],[0*scaleX, 1060*scaleY],[20*scaleX, 1080*scaleY],[1900*scaleX, 1080*scaleY],[1920*scaleX, 1060*scaleY],[1920*scaleX, 20*scaleY]];
        myShape.inTangents = [[11.045654296875*scaleX,0*scaleY],[0*scaleX,0*scaleY],[0*scaleX,-11.457153320312*scaleY],[0*scaleX,0*scaleY],[-11.457153320312*scaleX,0*scaleY],[0*scaleX,0*scaleY],[0*scaleX,11.045654296875*scaleY],[0*scaleX,0]*scaleY];
        myShape.outTangents = [[0*scaleX,0*scaleY],[-11.457153320312*scaleX,0*scaleY],[0*scaleX,0*scaleY],[0*scaleX,11.045654296875*scaleY],[0*scaleX,0*scaleY],[11.045654296875*scaleX,0*scaleY],[0*scaleX,0*scaleY],[0*scaleX,-11.457153320312*scaleY]];
        
        myShape.closed = true;
        myProperty.setValue(myShape);
        app.endUndoGroup(); 
    } 
}

Thanks again!

Liam

[Mathias Moehl]

AVLayers also have a width and height property:

https://ae-scripting.docsforadobe.dev/layers/avlayer.html#avlayer-width

Hence, you should be able to simply use your layer's width and height instead of the one of the comp.

SourceRect is helpful for text and shape layers, since those have the same size as the comp, but their actual content might have a different size (and change over time by keyframing the text or animating shapes).

[LiamCarlin]

Ah that's great. I had totally overlooked layer.width. Thanks Mathias!

[Mylenium]

You can simply proportionally scale based on existing info as Mathias suggested, you can implement the actual Bèzier formula or given the cubic nature of the curves just assume that around 33.33 percent tangent length makes for a perfect circle and thus multiply your values with 0.3333 or something. Whatever works best for you.

Mylenium