Skip to main content
.Ava.
.Ava.
Known Participant
July 7, 2023
Question

How can I make sampleImage sample a mask instead of a set radius?

  • July 7, 2023
  • 3 replies
  • 883 views

Someone here kindly wrote me a script, that I've tweaked with ChatGPT.

 

The role of my script is to scan a radius for brightness, scrub through the video and detect the lightest and darkest frame in that video, and find the mid-point between those two brightnesses. I also make it do a couple of other things that aren't so relevant to this.

 

The thing is, I want it to sample inside a mask I have, instead of in a set radius. I tried doing this through ChatGPT, and it did end up using the mask as a reference, but it only sampled a single pixel from the center of the mask, when I need it to sample all the pixels in the mask. As soon as I tried to make it so that it sampled all the pixels, it was laden with error messages, and became stupidly complicated.

 

TL;DR: How can I make the sampleImage sample all the pixels (not just one) in a specified mask, instead of a set radius?

 

 

 

function getLightestAndDarkest() {
    var frameScan = 300;
    var sampleRadius = 500;
    var brightnessThreshold = 0.1;
    var comp = app.project.activeItem;

    if (comp && comp instanceof CompItem) {
        var layer = comp.selectedLayers[0];
        var pt = [layer.width / 2, layer.height / 2];
        var lightExpr = "s = sampleImage([" + pt[0] + "," + pt[1] + "],[" + sampleRadius + "," + sampleRadius + "],false,time);\rrgbToHsl(s)[2];";
        var slider = layer.property("Effects").addProperty("ADBE Slider Control");
        slider.property("ADBE Slider Control-0001").expression = lightExpr;
        var sliderMin = 1;
        var sliderMax = 0;
        var frameMin = 0;
        var frameMax = 0;
        var frameMid = 0;
        var f = 0;
        var t = 0;
        var curVal;
        var midVal;
        var skinGlowLayer = comp.layer("Skin Glow");
        var lightestSlider = skinGlowLayer.effect("Lightest Frame")("Slider");
        var darkestSlider = skinGlowLayer.effect("Darkest Frame")("Slider");

        // Remove existing markers with specified names
        var markerProperty = layer.property("Marker");
        var numMarkers = markerProperty.numKeys;
        for (var i = numMarkers; i >= 1; i--) {
        var markerComment = markerProperty.keyValue(i).comment;
        if (markerComment.indexOf("Lightest") == 0 || markerComment.indexOf("Darkest") == 0 || markerComment.indexOf("Mid") == 0) {
        markerProperty.removeKey(i);
        }
    }


        while (t < comp.duration) {
            curVal = slider.property("ADBE Slider Control-0001").valueAtTime(t, false);
            if (curVal > sliderMax && curVal > brightnessThreshold) {
                sliderMax = curVal;
                frameMax = f;
            }
            if (curVal < sliderMin && curVal > brightnessThreshold) {
                sliderMin = curVal;
                frameMin = f;
            }
            f += frameScan;
            t = f * comp.frameDuration;
        }
        midVal = (sliderMax + sliderMin) / 2;
        f = 0;
        t = 0;
        var closestVal = 1;
        while (t < comp.duration) {
            curVal = slider.property("ADBE Slider Control-0001").valueAtTime(t, false);
            if (Math.abs(curVal - midVal) < Math.abs(closestVal - midVal)) {
                closestVal = curVal;
                frameMid = f;
            }
            f += frameScan;
            t = f * comp.frameDuration;
        }
        slider.remove();
       
        lightestSlider.setValue(frameMax);
        darkestSlider.setValue(frameMin);

        var marker = new MarkerValue("Lightest - " + frameMax.toLocaleString());
        layer.property("Marker").setValueAtTime(frameMax * comp.frameDuration, marker);
        marker = new MarkerValue("Darkest - " + frameMin.toLocaleString());
        layer.property("Marker").setValueAtTime(frameMin * comp.frameDuration, marker);
        marker = new MarkerValue("Mid-point - " + frameMid.toLocaleString());
        layer.property("Marker").setValueAtTime(frameMid * comp.frameDuration, marker);
        alert("Lightest frame = " + frameMax + "\nDarkest frame = " + frameMin + "\nMid-point frame = " + frameMid);
    } else {
        alert("No comp active.");
    }
}
getLightestAndDarkest();

 

This topic has been closed for replies.

3 replies

R
Community Expert
Rick GerardCommunity Expert
Community Expert
July 9, 2023

Try this.

  1. Duplicate the face layer and name it Face Track
  2. Rotoscope the face to create a layer with only the face
  3. Use Mocha AE or AE's tracker to motion stabilize the face by tracking a feature on the face like an eye or the nose so the face does not move
  4. Blur the face using Gausian Blur with Repeat Edge Pixes turned on
  5. Now you have a layer containing almost all of the pixels on the face that have been averaged that does not move.
  6. Place a null in the center of the motion stabilized blured face and name it Face Center
  7. Measure the size of the blured face and give it a little padding - The face I tracked was about 340 wide and 630 high*
  8. Add this expression to the layer you want to capture the data from the face

 

 

smpl = thisComp.layer("Face Track");
cntr = thisComp.layer("Face Center");
rds = [340, 630];
rgb = smpl.sampleImage(cntr.position, rds/2, true, time);
lum = rgbToHsl(rgb)[2];

 

 

Now you can turn off the "Face Track" layer (or make it a guide layer) and retreive the average luminance value of the face to drive the glow effect on the skin. 

 

Instead of a bunch of if statements I would add a mask to the original layer and use the mask for compositing options. If you want the glow on the entire layer then just double click the Rectangle tool to add the mask to the whole layer. If you name the mask Glow Effect then you could add an expression like this to the Glow Effect mask:

 

 

smpl = thisComp.layer("Face Track");
cntr = thisComp.layer("Face Center");
rds = [340, 630];
rgb = smpl.sampleImage(cntr.position, rds/2, true, time);
lum = rgbToHsl(rgb)[2];
t = lum;
tMin = .2;
tMax = .7;
value1 = 20;
value2 = 100;
easeIn(t, tMin, tMax, value1, value2)

 

 

tMin would be the luminance value that starts to change the overall Glow change, tMax would be the luminance value where you want the maximum glow. value1 determines the lowest percentage of the Glow effect applied to the layer and value2 lets the full Glow effect be applied to the layer. 

 

If you wnat to limit the glow to only the face you can use a copy of the Face Mask you already tracked and expand or soften it as needed.

 

By using a mask as compositing options and controlling the opacity of the effect by adjusting the opacity of the Composition Option Mask you can manipulate the entire intensity of the effect with a single expression driving a single value. Changing ease to easeIn or easeOut or linear will give you control over the ramping of the effect as the average luminosity of the face changes.

 

This looks like the simplest solution to your design problem. 

* Note: I measured the face by just looking at the Infot Panel as a moved the mouse around and made a mental note of the X and Y values. You could open up the ruler if you wanted to make calculations easier and drag the corner to the top left corner of the blured area. I hope this helps.

.Ava.
.Ava.Author
Known Participant
September 2, 2023

Oh wow. Thanks so much, Rick. You're always so helpful here. This is the first time I'm seeing this.

 

I managed to resolve my issue. Appreciate it.

Mylenium
Mylenium
Legend
July 8, 2023

There isn't a single piece of code even referencing a mask, so how would the sampling even be constrained? That aside, genuinely sampling something within a mask would be a major exercise. You'd need to re-create the entire code for masks to determine the boundaries and then test the sampleImage() for whenever it intersects with one of those edges. That's a lot of complicated slow code and even a simplified version would take considerable time to process. Even Rick's suggestion doesn't make a lot of sense if you genuinely want to sample inside a mask or another custom area. His simple radial sampling could be done with just placing a Null or Point control and calculate from there. No need to even use the mask. Anyway, I think you are completely in over your head and in none of your posts so far have actually explained what the end goal of your convoluted approach is supposed to be. Just finding out some median color can't be it. That could be determined and manipulated by applying some effects just as well.

 

Mylenium

.Ava.
.Ava.Author
Known Participant
July 8, 2023

I'm aware there's no code referencing a mask. That's specifically why I'm here, because I want there to be code referencing a mask, but I don't know how to achieve that.

 

The reason I want to find the lightest and darkest frames, is so I can quickly and easily just run this script, and use the lightest and darkest frames as reference points to change the glow settings.

 

I have a glow effect for my skin, but the lighting on my face changes throughout the video. At times, the glow effect was too weak, and others too strong.

 

The reason I wanted to create a script for this is because I'm using this composition as a base template for future videos, so that in the future, I can quickly identify the lightest and darkest parts of the video, and use an expression for the glow effect to adjust between those lighting conditions.

 

The reason I want it to be within a mask is because, behind me, there are multiple lights. When the radius is scanning a circular area, it includes those lights, and depending on how I move throughout the video, sometimes revealing these lights, sometimes hiding them. The lights would significantly alter the average brightness, not giving me as accurate results as I want, because I want it to assess the brightness of mostly just my face. The mask excludes these lights.

Dan Ebberts
Community Expert
Dan EbbertsCommunity Expert
Community Expert
July 8, 2023

I can't think of a practical way to get there.

R
Community Expert
Rick GerardCommunity Expert
Community Expert
July 8, 2023

You cant directly use a mask, you have to set a value for the radius you want to samaple. You can create an oval, but you have to define the x and y valeus for that sample area.

 

If you must use a mask you could create an eliptical mask, apply the Create Nulls From Paths/Nulls Follow Points script to generate 4 nulls, and if the Ellipse had the points at 12, 3, 6, and 9 o'clock, you could calculate the distance between 12 and 6 and 3 and 9 to get a radius value for the sampleImage(point, radius = [Null-12.position[0]/2 - Null-6.position[0]/2, Null-9.position[1] - Null-3.position[1]], postEffect = true, t = time) calculations. That's a lot more work than just using a null to define the center of the radius (you can make it an oval).

 

I think you have way over complicated the rest of your script. 

 

I'm not sure what you are trying to drive with the luminance value. If you explained what the design goal was maybe we could help you simplify the project. 

Terms & ConditionsAccessibility statement
Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices