Retrieve current frame as float data within UserChangedParam()

[yergin]

Hi Community,

I posted almost the same question in the Premiere Pro discussions and after no activity there, I thought I should post it here as this might be a better place for questions concerning the AE Plugin SDK.

I'm creating a video effect for Premiere Pro using the AE Plugin SDK and I'd like to analyse the current frame when the user pushes a button and then set some parameters based on the analysis.

Firstly, I managed to retrieve the current frame using PF_CHECKOUT_PARAM from within UserChangedParam() and set other parameters from the analysis however the returned layer is formatted as PrPixelFormat_ARGB_4444_8u and I need higher precision than 8-bit for the type of analysis being done, simply converting to float won't do it as I'm working from 10-to-16 bit log and linear footage.

Secondly, when I add the effect to an adjustment layer which does not begin at start of the timeline, the returned frame is not the frame under the cursor, since inData->current_time is apparently the current time relative to the beginning of the adjustment layer but param.u.ld holds data to the frame at inData->current_time relative to the beginning of the timeline.

I'm setting the following flags in GlobalSetup():

    out_data->out_flags = PF_OutFlag_DEEP_COLOR_AWARE | PF_OutFlag_KEEP_RESOURCE_OPEN | PF_OutFlag_WIDE_TIME_INPUT;
    out_data->out_flags2 = PF_OutFlag2_FLOAT_COLOR_AWARE | PF_OutFlag2_SUPPORTS_THREADED_RENDERING;

    if (in_data->appl_id == 'PrMr') {
        auto pixelFormatSuite = AEFX_SuiteScoper<PF_PixelFormatSuite1>(in_data, kPFPixelFormatSuite, kPFPixelFormatSuiteVersion1, out_data);

        (*pixelFormatSuite->ClearSupportedPixelFormats)(in_data->effect_ref);
        (*pixelFormatSuite->AddSupportedPixelFormat)(in_data->effect_ref, PrPixelFormat_BGRA_4444_32f);
    }

And this is a code snippet from UserChangedParam():

    PF_ParamDef param;
    auto err = PF_CHECKOUT_PARAM(inData, ParamIndex::kInput, inData->current_time, inData->time_step, inData->time_scale, &param);
    ...
    auto layer = &param.u.ld;
    ...
    AEFX_SuiteScoper<PF_PixelFormatSuite1> pixelFormatSuite(inData, kPFPixelFormatSuite, kPFPixelFormatSuiteVersion1, outData);
    PrPixelFormat format = PrPixelFormat_Invalid;
    pixelFormatSuite->GetPixelFormat(layer, &format);
    ...
    if (format == PrPixelFormat_ARGB_4444_8u) {
        cv::Mat src(layer->height, layer->width, CV_8UC4, layer->data, layer->rowbytes);
        ...
    }
    PF_CHECKIN_PARAM(inData, &param);

Any hints as to how to obtain the current frame with float precision on a button press or how to deal with the time offset of the adjustment layer would be appreciated!

[shachar carmi]

take a look at AEGP_RenderAndCheckoutFrame from AEGP_RenderSuite. it allows you to fetch any project item at any render settings.

obtain your effect's layer using AEGP_GetEffectLayer and then the layer item using AEGP_GetLayerSourceItem.

[yergin]

I tried to respond directly but it would not post, please see my response below. Thanks!

[yergin]

These "post flooding errors" are quite annoying, I can't seem to just reply to posts... I just read in the SDK docs that "No AEGP calls are supported by Premiere Pro."

[yergin]

For get a 32-bit frame, I've tried calling EffectWantsCheckedOutFramesToMatchRenderPixelFormat() in PF_Cmd_GLOBAL_SETUP but it's not made any difference, I'm still getting an 8-bit frame.

[yergin]

 For the adjustment layer issue, I might see if I can deal with the time offset using callbacks from PF_UtilitySuite.

[yergin]

Thanks for the quick response!

I tried the approach you mentioned but I got stopped quite early on:
*AEGP_RegisterWithAEGP(NULL,
, &id)* is not working for me from Premiere which I seem to
need for the render options to *AEGP_RenderAndCheckoutFrame()*. I also
tried just *AEGP_GetEffectLayer(inData->effect_ref, &layer)* from
*userChangedParam()* and that returns error 515 (not sure what that means).

I would love to use the AEGP callbacks in my effect, they seem very
powerful!

Have you had any luck making these calls from within Premiere Pro before?

[shachar carmi]

you can skip AEGP_RegisterWithAEGP entirely. just pass a NULL to whatever function is asking for the id.

alas, i think none of the AEGP suites work in premiere...

[yergin]

Yes, you are right, I just read in the SDK docs that "No AEGP calls are supported by Premiere Pro" so I'll need to find another way. 

[AnmolM]

In the sdk example you will find one that gives ARGB 8-bit and 16-bit frame for PPro.. Use it with appropriate comp settings to get 16-bit data.

I don't think PPro offer float data using AE SDK.