Skip to main content
Participating Frequently
September 26, 2023
Answered

Is there a way to setup a plugin to anycrhronnously receive video frames from a specified pipe?

  • September 26, 2023
  • 1 reply
  • 1662 views

I am trying to modify a sample plugin to rwite frames to a pipe and anycrhronnously receive frames from that same pipe. However, I cannot find a sample plugin that anycrhronnously receives frames. Any idea on how I can achieve this? 

This topic has been closed for replies.
Correct answer shachar carmi

@shachar carmi Also, is there a sample plugin that resizes the ouput frame, for example from 1x to 2x? 


take a look at the "Resizer" sdk sample project.

1 reply

Community Expert
September 26, 2023

i'm assuming that by "pipe" you mean an external source as opposed to AE being the source.

if that is indeed the case, then the issue if reconciling the random AE render calls, and the async pipe.


for the async pipe, you can setup a separate thread (regular c++ thread, nothing to do with the AE SKD), and have it read/write data to some pre-allocated memory. with a mutex, you can access that memory from both the async pipe and AE's calls. if you want to "force" AE to talk to the pipe thread asap, you can use AEGP_CauseIdleRoutinesToBeCalled(). (look it up in the docs)

 

as for sending and fetching frames, this is a bit of a design challange.

ae would call your plugin to render at some pretty random times, and in not in sequential order. if you want to send away whatever was just rendered, that's pretty straight foreward. lock the mutex and write to the pipe buffer.

however, if your plug-in requires some external frame that is still unavailble at the time of the render call... eeesh... now you need to choose between a couple of bad options: either stalling until the async thread fetches the frame, or telling AE the render failed, which would terminate the current render/preview.

so, in regards to using external async-gotten frames in an AE render, you need to plan the work scheme well so that you would minize occurences where not all required resources are available to the plugin at the time of a render call.

Participating Frequently
September 26, 2023

Thanks @shachar carmi, that is very helpful.

To explain better, I have an ML model which requires 3 frames (1 before current frame, 1 after) to process output for a single frame. Currently, the way the "engine" works, it expects frames to be read from a file in sequential order and fed into the engine. There is a pipe set up to listen to output. Once it starts rendering output, the callback handler is called for each frame rendered. Input is continously fed into the engine until it reaches the end of the file. This is why I say it works more like an async pipe than a synchronous render for each frame.

Today I have some C++ code that can do this in a standalone program. I am trying to see how I might be able to make an Effect plugin that uses the same engine. Do you have some ideas about how to do this?

Community Expert
September 26, 2023

well... AE's rendering pipeline is built around the logic of "random frame access". AE allows the users to jump around the timeline freely, rendering the current frame only whithout needing to fetch all previous frames. furthermore, now that AE has multiple render threads, the render calls are likely to not happen in sequental order.

so... we're off to a bad start.

 

you can have an AEGP plug-in monitor the project and detect changes in the input layers of each of your plug-in's insances, and then fetch source frames using AEGP_CheckoutOrRender_ItemFrame_AsyncManager or AEGP_CheckoutOrRender_LayerFrame_AsyncManager as not to stall the UI. i think that as a next step i'd write the resulting frames into some cache folder to fetch on demand. (some headache to manage that cache to avoid bloat, but not the end of the world)

in the case where the user jumped past the currently cached result frames, THEN you can stall the render until the caching catches up. in that scenario you can do it synchronously to speed thigs up.

 

sorry, i don't know of an elegant solution to this conflict of system designs between AE's "frame independance" and your "sequence dependent" process.