GPU Out of Memory (Error 517) with checkout_layer_pixels in Stateful Particle System

Hi everyone,

I am working on a stateful GPU Particle System plug-in using the AE 2022 SDK. Because the plug-in is stateful, every particle's state at frame n depends on its state at frame n-1.

I am using a caching technique that stores previous states, which works perfectly during a standard serial preview. However, if the user jumps randomly to a later frame (or to the end of the timeline), the cache is invalidated, and I have to run a naive loop through all previous frames to calculate the current state.

During this loop, I need to sample a layer that might be animated. This requires checking out the layer for every frame within the loop. My current approach is:

Call checkout_layer inside PreRender().

Inside SmartRender(), for every cycle of the frame-calculation loop:

Call checkout_layer_pixels().

Process the layer data.

Call checkin_layer_pixels() to free the memory.

The Issue:
When testing with a large layer size, the plug-in crashes. After debugging, I found that the GPU runs out of memory. Specifically, checkout_layer_pixels begins returning error code 517. It appears that checkin_layer_pixels() does not free the GPU VRAM immediately during the loop; it only releases the memory after the entire SmartRender() call concludes.

My Questions:

What is the proper way to force the GPU memory to free inside every cycle of the loop?

How do established plug-ins (like Trapcode Particular) manage heavy layer sampling across multiple frames without hitting this VRAM limit?

Any insights or workarounds would be hugely appreciated!

Best regards,
Haytham Magdi