How can I use a difference matte to change the opacity of another layer?

[.Ava.]

Heya.

I'm really not very good at AE at all, so I'm not so good with the advanced terminology.

I have flameless candles in the backdrop of my static shots, and these static shots often last 45+ minutes. I have PNGs of lens flares that correspond to the candles. I love how it looks, but I notice that when the candles are momentarily obscured (often things like my hands or arms moving in front of them), the PNGs of the lens flares still remain, which looks jarring and unnatural.

I would like it to be such that when the candles are obscured, the opacity of the lens flares dynamically, automatically lower in opacity, proportionate to how obscured they are. So if they are partially obscured, the opacity of the lens flare would come down, but not entirely.

I would like this to be automated, as this will be what I'm doing for every video I make.

Ideally, I would like to be able to just mask out the small area of the candles and make AE just focus on the changes to that area, so it's aware of when it changes, and when it should modulate the lens flare opacity.

What I've been trying to do is sample pixels inside a certain radius, and changing the lens flare opacity based on luminance changes in the sample radius. This has been flawed though, because there are times where my hand has been as bright as the candle, and so it hasn't properly registered the changes.

I've heard that a difference matte might work? But I'm not experienced enough to be sure. I figured that I could take a freeze-frame of a frame where the candles are totally obscured and somehow find a way to make AE lower the opacity the more different the current frame is compared to the reference freeze-frame.

Thanks in advance.

[Warren Heaton]

This will be much easier with a lens flare effect that supports what is called "an occlusion layer".

You could use the trial version of Sapphire Lens Flare to see how this works. 

https://borisfx.com/documentation/sapphire/ae/lensflare/

[.Ava.]

Thanks so much! However, I went into labor when I saw the price of the license for the software.

[Dan Ebberts]

It would help to see what you're using so far. Is it a sampleImage() expression like this:

L = thisComp.layer("Candle Comp");
samplePoint = [448,506];
sampleWidth = 20;
sampleHeight = 40;
c = L.sampleImage(L.fromComp(samplePoint),[sampleWidth,sampleHeight]/2,false,time);
linear(rgbToHsl(c)[2],0,1,0,100)

[.Ava.]

Hi, Dan. This is what I'm doing so far:

I have 3 layers:

- Lens flare: Just a shape layer of a red circle as a placeholder for where the lens flare PNG will be.

- Base Video: Just the normal video of me talking.

- Candles: A pre-comp with a mask around the candles. The mask is named "Candles". It's a pre-comp that also contains a layer named "Reference Frame", which is a time-remapped freeze-frame of a frame where the candles aren't obstructed, and I have another layer, also named Video, which is just the normal video, but with a Difference Matte applied, with the source being the Reference Frame, so it detects the differences between the regular video and the freeze-frame.

As stated, in the Candles pre-comp, the differences between "Video" and "Reference Frame" are displayed in a black and white matte, so when my hand goes over the candles, the ordinarily, mostly black candles, are obscured by my white hand.

Right now, I'm using an expression to sample the candles, so that the more white the area is (meaning that there is more obstruction of the candles), the lower the opacity of the Lens Flare layer drops.

Here is the expression:

// Variables - Customize as needed

// Layer and mask settings
candleLayerName = "Candles";   // Name of the candle layer
maskName = "Candles";          // Name of the mask on the candle layer

// Slider controls from the Lens Flare layer
onThreshold  = effect("On Threshold")("Slider");   // Lower brightness bound for full opacity
offThreshold = effect("Off Threshold")("Slider");  // Upper brightness bound for zero opacity
maxOpacity   = effect("Max Opacity")("Slider");    // Maximum opacity value

// Calculate the mask's bounding box for sampling

candleLayer = thisComp.layer(candleLayerName);
mask = candleLayer.mask(maskName);
maskVertices = mask.maskPath.points();

xMin = yMin = Number.MAX_VALUE;
xMax = yMax = Number.MIN_VALUE;

for (var i = 0; i < maskVertices.length; i++) {
  var vertex = maskVertices[i];
  xMin = Math.min(xMin, vertex[0]);
  yMin = Math.min(yMin, vertex[1]);
  xMax = Math.max(xMax, vertex[0]);
  yMax = Math.max(yMax, vertex[1]);
}

sampleWidth = xMax - xMin;
sampleHeight = yMax - yMin;
sampleSize = [sampleWidth, sampleHeight];
samplePoint = [xMin + sampleWidth/2, yMin + sampleHeight/2];

// Sample brightness within the mask area

hsl = rgbToHsl(candleLayer.sampleImage(samplePoint, sampleSize, true, time));
brightness = hsl[2] * 100;  // brightness from 0 to 100

// Determine opacity based on brightness thresholds

// Initialize opacity level
opacityLevel = 0;

// If brightness is below the On Threshold, use max opacity
if (brightness <= onThreshold) {
  opacityLevel = maxOpacity;
} 
// If brightness is above the Off Threshold, set opacity to 0
else if (brightness >= offThreshold) {
  opacityLevel = 0;
} 
// Otherwise, interpolate between thresholds
else {
  // Map brightness from [onThreshold, offThreshold] to [maxOpacity, 0]
  var range = offThreshold - onThreshold;
  var positionInRange = brightness - onThreshold;
  var percentage = positionInRange / range;
  opacityLevel = maxOpacity * (1 - percentage);
}

opacityLevel;

[Dan Ebberts]

Thanks for the explanation. I think I understand the issue now, but I can't think of a better approach than what you've tried. Maybe someone else will chime in.

[.Ava.]

Thanks, Dan!

And honestly, if there are no better ways, I may just settle with this one. I was just hoping to find a more efficient way computationally and perhaps something more intuitive to implement.