Adding Fire Effect Behind an Occlusion Area in a Video

[Nickwhy90]

Hello there,

I've been busting my head the last 2 days on a particular problem while trying to turn a daytime shoot of a house into a nighttime scene with a burning fire from the windows. I'm having trouble keeping an occluding wooden joist on the porch of the second floor in front of the fire coming out of the window behind it. I made a track with Mocka there (tried without AE's regular mask as well) on both the window and the area with the joist (and the facade  below it) and used the trackings to create a mask and for the joist area so that the fire doesn't appear before it. However, when I add the glow effects on the fire, you can see the outlines of the mask and it ruins everything. I tried all sorts of combinations of the layers and the assets to "trick" the damn thing, but couldn't figure it out. I also used Mocha to track the other window and the bushes and used the tracking for the larger window fire, but I don't seem to be having a problem there. Here's  a screenshot:

Since there are quite some tweekings across the layers, I'm sending a link with the zipped project so you can check it properly on your own. I'd be very grateful if someone gives me a tip as to what might be wrong and how to correct it, so I'll know how to do it properly in the future when I have such situations: https://we.tl/t-7ToYLSb4cB

Best,

Nik

[Mylenium]

Your renderer is set to Cinema 4D which doesn't support blending modes and other features. You need to use the Standard 3D renderer.

Mylenium

[Nickwhy90]

Hi Mylenium,

Thank you for writing in the thread.

I switched to Classic 3D, but nothing changed.

[Mylenium]

Pre-compose the two layers and assign the shadow casting item as an inverted track matte for the fire in the pre-comp, then apply the glow in the parent composition.

Mylenium 

[Rick Gerard]

You don't have enough copies of the fire or enough layers for track mattes. There is no way to composite a realistic-looking fire coming from a window shot with just one fire layer. You'll need at least three and maybe as many as 5 different copies of the fire to get a realistic composite. 

The house also should be color corrected to Day For Night, pre-composed, and probably even rendered before starting the composite. 

Find an image of a fire in a window at night and carefully examine the way the fire lights the scene. You should always be comparing a VFX shot that has fire or explosions with an image of the real thing.

I think you need at least the following layers. 

A copy of the fire element with a track matte for the window frames that are significantly blurred and maybe have a glow effect applied, much lighter than the original fire with the blend mode set to Add and opacity adjusted to give a glow to the surface exterior surface of the house. The matte may also have to be animated as the flames lick around the edges of the fire.

A slightly blurred copy of the window matte to use as a softened track matte for another copy of the fire element layers to give an additional glow to the edges of the window frames. This matte would also have the main window areas filled in, so you end up with just the edges of the fire wrapping around the window frame edges. This layer is also set to Add or Screen and opacity adjusted. 

Another copy of the fire element with a hard-edged matte for the window frames so you have a clean copy of the fire just inside the window. No flames extend outside the window frames.

Another copy of the fire element with the center of the window cut out that you can lay on top of a copy of the house with the other copies of the fire so you can stack up the inside fire, the outside fire, the glow from the fire, some smoke, and the other elements to build a realistic fire effect in the shot. 

When you have a shot looking pretty good, pre-compose everything again and apply some final color grading to the shot. I would use at least two copies of Lumetri with masks to finalize the shot. The last time I did a similar shot, there were a total of about 20 layers and masked color correction layers in the final comp when you expanded the flowchart. 

[Nickwhy90]

Hi again,

I still need some help with another problem - I'm talking about the bushes that occlude the balcony as well. I don't mind them being cut outso they don't occlude the fire behind, as it is currently, but as the camera moves and the bushes are going below the balcony, you can see a branch keep sliding out below the pre-composed fire+ wooden joist occlusion and it's just not right. There's a similar situation with the larger window fire, as there's a small branch that occludes it in the beginning as well, butr it's much less noticable.

Current project video: https://we.tl/t-qJ4YrLHC9l

I looked into a way for removing the branches occluding the balcony for good via the rotoscope brush as it's the best way to select them witohut cutting the space between the leaves and branches, but I couldn't do it. I saw that you can try this with the inverted selection and the content aware function but it doesn't work in this case. I also tried the auto-trace function to create masks out of the roto selection and subtract them, but that doesn't work either.

Can you give me any tips as to how can I achieve this?

Best,

Nik

[Rick Gerard]

That is going to take some hand masking. I would use Mocha AE to do the rotoscope. You can find some good information on the workflow here.

[Nickwhy90]

I tried, but it didn't seem to get the job done. It removed the branches in the beginning of the shot, but the replacement content keeps sliding down when the camera moves. Here's the result: https://we.tl/t-Gkd9x2GUyH

I guess I picked a too difficulut shot to learn over...