Parameter-based arbitrary data reallocation in C++ effect plugin

[qdin]

Hi,

I’m writing a C++ effect plugin with a UI similar to AE’s Mesh Warp. So I have parameters “rows” and “cols”, and then a mesh of points, which I'm storing as arbitrary data. The mesh should be resized (and perhaps interpolated) in reaction to changes to the “rows” and “cols” parameters.

I’m trying to figure out the best way to do this, and have a few questions to help sort things out. 

1. The ‘params’ function parameter is usually null inside of my HandleArbitrary() call. I can call checkout/checkin on the param, but this seems to work only when in_data->effect_ref is not null. Is there a correct way to handle this, or do I just explicitly check (id_data->effect_ref != nullptr) before checking out params?
2. 'rows' and 'cols' are not animatable. Why can I only access them with checkin/checkout out?
3. Since effect params are not always available through the ‘params’ function parameter, or even by checking them out, it seems like I should store values like ‘rows’ and ‘cols’ in my arbitrary data structure. But then that means that I have to do another level of allocation, like allocate a separate array of points and store a ref inside my top-level arb data structure. If I do store references to other structures, do I have to allocate/free/lock/unlock that memory with the handle suite as well? 
4. If I am allocating my own ‘substructures’ (as in 3 above), where should I be doing this? I’m mainly getting COPY_FUNC calls and rarely any NEW_FUNC calls. Do I need to notify AE that I need to reallocate?
5. Can the above reallocation be done outside of a HandleArbitrary() call? For example in UserChangedParam()? 

Thanks in advance!

[shachar carmi · Accepted solution]

before we dive into specific detail, there's something you need to consider. to explain it, let's review some general behavior of arb params.

the plug-in is a functional skeleton, that AE sends any applied instance's data to travel through. there's no one place where the whole of an instance's data exists. AE calls upon the sekelton with the relevant instance data and a command to process the data according to.

however, sometimes AE uses the skeleton to process data without it being associated with a specific instance. arb params are processed that way. AE calls the skeleton with some arb data to either copy or interpolate, without providing any info to which instance that data came from, at what time or where the result is going (i.e. to render, to the clipbaord, for pasting, ect..)
so arb calls usually (if not always) do not have an effect ref, available sequence data or any associatable info to any specific instance.

so what does all that mean?

it means that a design where in order to provide meaningful arb data you also need to get specific param data from other sources, is goung to have a really hard time doing that within AE's arb param handling scheme. ideally, the render function (or any other function) would fetch the raw arb data and other relevant pieces of data which are relevant and are available during a render call (or any other call), and process the data there instead of during the arb call.

is that applicable to your situation?

[qdin]

Hi Shachar,

Thanks for the lightning fast response!

If I understand you correctly, you're saying that the arb handler is kind of a "stateless" function, i.e. when you're inside the function body there's usually no concept of an instance (or even system state) -- you only have the data that's passed through the function params on this particular call.

And that's why effect_ref is usuall null (and checkin/checkout rarely work) -- because you're just supposed to deal with whatever piece of data AE gives you without being able check state outside of that data.

If that's what you're saying, then I think the last part of what you wrote (which, yes, I believe is applicable) answers most of my questions (and renders the rest irrelevant 🙂 ) -- basically that in UserChangedParam() (probably) I should reallocate and store any allocation-related info (rows, cols, etc) in my arb structure so that HandleArbitrary() can process it without queryng system/instance state.

Do I have that right?

[shachar carmi]

yes, you understood me correctly. 🙂

[shachar carmi]

however, given your solution you shod consider the scenario where a user copies the arb data from one instance and pastes onto another where the grid params are different. i'm not sure you'll get a "user changed param" call on such an occurrence.

[qdin]

Gotcha, thanks - I'll keep that in mind. Maybe the supervise flag'll do it? I'll take a look.

I think the only piece remaining is the end of my question 3 -- if I have multiple levels of allocations, 

for example, I store a handle/pointer in my main arb data structure to my point data, does this 'second-level'

data also need to be allocated/deallocated/locked/unlocked with the handle suite (vs normal new/delete)?

[shachar carmi]

well... that depends on what you want to end up with.

arb params get flatten and unflatten calls (flat = serialized = all the data is in one continuous chunk of memory. unflat = deserialized = data contains pointers to other chunks of memory). your plug-in is required to produce serialized and deserialized data accordingly.

i don't remember if the copy call is always preceeded by a flatten call and followed by an unflatten one...

so assuming you get a copy call with unflat data as an input. what does your implementation require? that the new copy point to the same piece of shared data, or to an independent copy of the data?

[qdin]

I think it should behave like a C++ object holding a vector of floats. So yeah, it should probably make a "deep copy", and not share the data. 

[qdin]

So if COPY_FUNC gets called after a copy-paste operation between two instances of my plugin, and, say, they have differently sized meshes, then the target's rows and cols will be updated, and the target's float/pt data will be reallocated, its data copied in from the source's. 

And similarly for flatten/unflatten.

Unless I'm missing something, it seems like that float/point data should never be shared across objects.

[qdin]

Anyway, this is plenty to run with. Thanks for all of your help!