Is there a limit to the number of masks on a layer (about 400?)?

[A.I.1]

Is there a limit to the number of masks on a layer? If there is, shouldn't it stop you from adding more (and give a relevant message)?

When I get to about 404 masks on a layer in the latest After Effects version 22.4.0 (on a PC) it stops responding and paints the composition layer tab with black instead of the layers. I tried switching from GPU acceleration (using RTX 3090) to software only and it still stopped responding around 404 masks on the layer.

Has anyone used a lot more than that on a layer and had it work okay? Is it an out of memory issue somewhere (either on the GPU or RAM?). I have 24 GB RAM on the GPU (it may not be allocating all of it in AE) and 256 GB RAM (though not all is available for AE).

Do I need to move some of the masks to another layer if there's too many on one layer? Should it be okay then? eg. so that there's no more than eg. 400 masks on each layer but it's okay if the composition has more (eg. would it be find hypotheically if one layer had 400 masks and the next also had up to 400 masks? (I shouldn't need that many on the 2nd layer I'm just wondering hypothecialy if it would work).

The layer the composition (3840x2160 25 fps) is using for the masks is another composition that's 3840x2160 pixels, 25 fps.

Basically I'm using the compostion with the many masks on the layer to remove many areas of the layer so I can then do a "content aware fill" for all those mask positions. Would it be better not using the other composition for the layer used for all these masks and instead rendering either a frame or a pre-rendered video of the other composition and using that for the layer with many masks? Would it then allow more masks on the layer than around 404?

[Mylenium]

Well, let's get the obvious out of the way: In such a scenario content aware fill likely wouldn't even work because your extensive masking would have removed so much content it couldn't make sense of what's left. That said, your whole approach sounds less than ideal regardless of the circumstances. Except for the most atrocious auto-trace I couldn't even imagine what one would do with 400 masks. In theory you should not have any limit on the number of masks or at least 999, but of course all that info has to be managed. Chances are that you simply reach some internal limit even when you don't genuinely run out of memory and your indirect approach may exacerbate matters since it forces AE to constantly re-render everything. There's a good likelihood you could have a few more masks if you applied them directly. Anyway, this simply sounds like a terrible workflow and in order to get anything done you possibly cannot avoid pre-rendering your footage with transparency, then re-importing and replacing it to add more masks. Again, to me everything sounds a bit insane. That said, if you provide screenshots and explain what you are actually trying to do we may be able to advise on alternate, better workflows.

Mylenium

[A.I.1]

Thanks.

Well, let's get the obvious out of the way: In such a scenario content aware fill likely wouldn't even work because your extensive masking would have removed so much content it couldn't make sense of what's left....Except for the most atrocious auto-trace I couldn't even imagine what one would do with 400 masks.

What the shot is is a timelapse shot with water and boats and a close land and buildings in the foreground and mountain/hill-type areas of land on the far sides of the water. It's the areas past the water, in the distance that I'm using these masks on. So the timelapse video was originally a shorter video (and has been lengthed a lot) and it had started in the evenining/sunset when there were lots of lights on all over the mountain type area. The person I'm doing it for wants it to seem like the shot lasts for a whole 24 hours and in the day time wants all the lights off. So the masks were to mask out most of the lights on the opposite side of the water where there's >400 lights.

I've added a screenshot of a section at 200% zoom from the 4K comp showing some of the lights I'm trying to remove to give an example.

I did try using adjustment layers in the main composition to remove them but that started slowing down things a lot. For the close parts of the shot I recreated things more precisely (like building walls) and added things like trees over areas (I don't know for sure if those are going to be okay) and for some lights in the distance I added cut out buildings over and I think a tree.

In theory keying might have worked to help remove the lights (help define where they are) but I couldn't make that work for this.

I don't think it needs to be 100% accurate. The content aware removal seemed to be doing an okayish job with subtract masks with 2 pixel feathering. I'm working with the 4K footage but the delivery will be at 1080p so that should make slight errors less visible. So if the content aware makes it look like whatever the background was roughly it could be okay (though when they're near buildings it probably needs to be more accurate than when they're just over a mountain side area with no buildings).

Thanks for your help. I'll try doing the things you suggsted. I wish Adobe would put proper error/warning type messages into the software and not have it just stop working properly when it runs past their limits though.

It's a bit of a really difficult thing to be asked to do though - turning off all lights in a whole scene/town type area. I mean it's a lot easier to add lights to a scene than it is to remove them. And it's a stationary camera shot.

In theory an AI software could be made to do something like this (turn an evening/sunset shot into normal daytime without any lights on). There's a demo of something Nvidia has done (edit: for changing seasons and day to night not night/evening/sunset to day) but it wasn't error-free and I don't think there's AI stuff available to the public that's built for that yet but maybe it could be in future.

[Mylenium]

Turning off lights just by ways of some magical AI entails a ton of extra functionality as in your simple cropped screenshots there are five different types of "light" - anything from indirect illumination to secondary halos. Far from trivial and any algorithm would have to account for that and calculate normal maps or re-create the scene with actual 3D geometry and then re-project the cleaned up textures. Could this be in AE? Maybe, but ultimately Nuke already does most of this and still requires lots of extra work, so chances are slim it will ever come and for that matter it would require more than single shots. And that is likely also the crux with your current project. Even if 100% realism isn't required you can work your fingers off with masking and relying on CAF. It might in fact be simpler to just chew through this image by image in Photoshop and using CAF and other Neural filters there. I really can't think of much else as apparently any other workflow would have required to plan for it beforehand while shooting the footage.

Mylenium

[A.I.1]

Thanks.

Turning off lights just by ways of some magical AI entails a ton of extra functionality... Far from trivial and any algorithm would have to account for that and calculate normal maps or re-create the scene with actual 3D geometry and then re-project the cleaned up textures.

Though if it was all done with AI it shouldn't really need the normal maps/3D geometry. In theory it should be able to learn without that. In theory I think you could pass it many eg. evening images and their corresponding midday images and the neural network through the different neural net layers should be able to learn what weights/settings best convert the evening image to midday.

eg. this is the demo of an AI converting between seasons in a video:

https://www.youtube.com/watch?v=9VC0c3pndbI&t=5s

This is a demo of day to night by AI on a video:

https://www.youtube.com/watch?v=mmj3iRIQw1k&t=3s

So maybe a similar thing could be used for converting sunset (with lights) to midday (without lights).

 Even if 100% realism isn't required you can work your fingers off with masking and relying on CAF. It might in fact be simpler to just chew through this image by image in Photoshop and using CAF and other Neural filters there. I really can't think of much else as apparently any other workflow would have required to plan for it beforehand while shooting the footage.

Thanks. I have Photoshop Elements on my old PC which has a heal brush like content aware fill - I don't have it on my newer PC. I have Affinity Photo on it but I don't think it has a good content aware fill equivalent. So I'll probably buy the latest photoshop elements if it has that (the healing brush - I don't think elements has the normal content aware fill) since it seems not too high price (I thought it was subscription - the Photoshop one is subscription / higher priced). So I could try that. Doing it the mask way in AE helps if I need to later edit things though (and might help make things more precise than painting with the healing brush). The changed areas need to be masked in some way to be able to be placed throughout the video too (though I could use a more basic simple mask of the whole area and make that part of the scene not change much throughout the video).

[Mylenium]

Well, you are making your own point: Not enough references, nothing to learn. That's the basic limitation of any current AI model since they are merely based on statistics. In order to even come close to solving your conundrum you'd need a few thousand similar shots at least for the algorithms to learn from and that is highly unlikely. How this is approached on the implementation side is an open question, of course, but I doubt a simple style transfer liek in your video would work. Light tends to bounce around and unless you correlate e.g. which light source casts which shine or shadow, this can never look particularly real. I'm pretty certain you would need some sort of geometry or an approximation with normal maps/ normal passes. Anyway, let's just settle on that there currently is no good way to handle your specific scenario even in a semi-automatic fashion in AE and you just have to grind through it the hard way.

Mylenium