GPU accelerated AE/Premiere Pro plugin development?

[Guest]

Hello All,

I'm the developer of a prototype video processing algorithm that is currently a standalone .NET executable.

I want to create an After Effects or Premiere Pro compatible plugin version of this algorithm.

My questions:

1) Should I use the After Effects SDK, or the Premiere Pro SDK to develop this plugin? Does an After Effects plugin run in Premiere Pro as well? Or, conversely, does a Premiere Pro plugin work in After Effects?

2) Can an After Effects or Premiere Pro Plugin tap into the power of the GPU? Do I need to learn CUDA/OpenCL for this, or does the Adobe SDK include its own GPU functions?

3) Which SDK makes it easier to create a multithreading-capable plugin, so that my plugin uses all CPU cores on a machine?

4) Is there a way to get a free developer version of After Effects/Premiere Pro, or do I have to buy the full price products to develop plugins for them?

Thanks for any help/answers!

[Guest · Accepted solution]

Hi there!

Let me try to answer some of these questions to the best of my knowledge. I am currently also in the process of developing GPU-accelerated AE plugins, so I had to deal with some of these issues and questions over the last months myself .

1) The AE SDK allows you to develop plugins for AfterEffects (naturally), but these plugins are - with some restrictions - also usable in Premiere Pro. Please download and check the AE SDK for more information, but from top of my head, I know that JS scripting and many processing suites are not supported in Premiere Pro and that Premiere Pro uses a different pixel layout than AE (BGRA vs. RGBA for example). So, an AE plugin can also work in Premiere Pro, if coded appropriately. What I did was checking within the plugin from what host it was called and then calling the appropriate functions. Also check out the AE SDK examples, there is for example the "Portable" and the "SDK_Noise" example for plugins that work across AE and PremierePro.
On a side note, the PremierePro SDK (afaik) only allows developing native PremierePro plugins that cannot be used in AE.

2) Sure a plugin can use the GPU - in that sense there is not much difference between a plugin and a standalone application. There is however no GPU-related example in the SDK, only a rudimentary and broken example for simple fixed pipeline OpenGL integration (example name "GLator"). But you can of course create your own OpenGL context (preferably a hidden window/renderbuffer) and interface the GPU any way you want. Keep in mind the switching of the context between native AE and your own GL context.
So yes, it is possible, but you have to do all the hard work yourself, the SDK merely provides an empty framwork and you have to code your own interface functions for it.

The way preferred by most is probably (as I said above) using OpenGL and then compiling GLSL shaders to run on the GPU. The main obstacle is getting your video input frame(s) uploaded to the GPU, so you have to read each pixel from the AE pixel buffer, convert it into a OpenGL texture, upload it to the GPU, then let the GPU do its magic and read back the renderer OpenGL renderbuffer pixel by pixel to the AE output buffer. Again, look into GLator example for an idea how to do this (though there are some slightly more effective methods).

3) AE SDK should be fine already for multithreading. Most function/callbacks for a plugin are not threadsafe themself in the SDK as they are distributed and called across CPU cores as needed by AE automatically. So when properly using non-blocking and threadsafe callback functions for pixel processing, like it is done in most AE SDK examples, you should be fine on that level I think. Should you need deeper control over the threads, you have of course to do it manually, for example by making one blocking render call and start your own threading from this (not that good in practice, but you should get the idea).

4) You have to contact Adobe and ask them for a NFR/developer version, but I don't know if they offer this for everyone. Besides that, there is a time-limited trial version that should also work with third-party plugins I think, so that might be a cheap way to test it out

Cheers,
Toby

[Guest]

Mdoepfer,

Thank you very much for the info... So, there is no real GPU functionality in After Effects/Premiere SDK? You have to do this through a GLSL shader?

Is there a GLSL example included in the SDK? I've never tried GLSL before, but if there is an example, then maybe I can figure it out...

Also, this "GPU accelerated Mercury Engine" thing Adobe advertises for Premiere Pro CS6... Is there a way to tap into this?

Or does Mercury Engine bypass the SDK, so you cannot write a plugin that uses it?

I'm still undecided whether to develop for AE or Premiere, and which software to buy from the local distributor...

If I develop for AE, there seems to be a chance that the plugin works in both AE and Premiere Pro.

Anyway, thanks for the info! The overall picture has become a little clearer for me now.