Question Regarding Temporal Smoothing and Frame Processing Order in AE/Pr SDK

[marks81797503]

Dear Adobe Teams,

I am developing custom filters for After Effects and Premiere Pro using the official Adobe SDK. From time to time, to prevent output artifacts such as flickering in the rendered sequence, I need to implement temporal smoothing—for example, combining results from consecutive frames.

From my understanding, both AE and Pr use a multi-threaded rendering engine, meaning that my Render function may be invoked simultaneously for multiple frames in the sequence (meaning called from number of host threads). Could you please confirm if this assumption is correct?

If so, I would like to know whether there is any mechanism or guarantee that Render calls are executed in frame order (i.e., frame 1, frame 2, frame 3, … frame N). This is critical for implementing temporal smoothing correctly, as it requires combining results from frame N with those from frame N-1.

Specifically:

Can I safely assume that the SDK will invoke Render in sequential frame order?

When processing of frame N is near completion, is it guaranteed that frame N-1 has already finished processing?

Any guidance on this matter would be greatly appreciated, as it would significantly influence how I implement temporal smoothing in my plugin.

Thank you in advance for your support.

Best regards,

[Kevin-Monahan]

Hello @marks81797503,

Thanks for the question. This is a user to user forum, so I can't guarantee an answer here from the team but I will let them know about it. Other developers do enter into our forum from time to time, as well.

As I understand it, you cannot assume that the Render function will be called in sequential frame order. After Effects uses a multithreaded rendering engine, especially with Multi-Frame Rendering (MFR) enabled in After Effects 2022 and later. This means multiple frames may be rendered concurrently across different threads, and not necessarily in order.

Since I am not an engineer or a product team member, I will let others describe on what can be done to work around the issue.  I hope the advice helps a bit.

Thanks,
Kevin

[marks81797503]

Hi Kevin,

Thanks for your reply.

I have a couple of questions regarding frame synchronization and buffer management for my effect:

1. Frame Identification: Is there any specific field or indicator within the in_data that would allow me to definitively identify the frame number in the sequence? I need this information to build a correct mapping.

2. Buffer Management Strategy: I'm a bit unclear on how buffer management works in this scenario. My understanding is that when Render() is called within my effect, I receive ownership of both the input buffer (for rendering) and the output buffer (for results). After Render() completes, I'm expected to return these buffers to the rendering engine.

Is it possible to "hold" these buffers in any way, or to force the Render() method to perform frame-ordered rendering? I need to wait for results from other frames before proceeding with certain operations.

Any insights you can provide would be greatly appreciated.

Thank you in advance.

[James Whiffin]

Hi Marks

If MFR is enabled (which it is by default), you can't guarantee frames will render in any order. I believe if MFR is off it will give you the frames in order and one at a time, but you'll want to double check. 

"both AE and Pr use a multi-threaded rendering engine, meaning that my Render function may be invoked simultaneously for multiple frames in the sequence (meaning called from number of host threads)." to my knowledge this is correct.