Create Puppet Pin programmatically

[MartinMH]

Hi all,

I'd like to ask whether is it possible to create puppet pins programmatically...

I have tried to copy stream structure ("ADBE FreePin3", "ADBE FreePin3 Mesh", "ADBE FreePin3 PosPin Position", ...) of manually created pins with

corresponding values, but when I create streams this way they are disabled (AEGP_DynStreamFlag_DISABLED - and greyed out in UI).

I suppose that the vertex index "ADBE FreePin3 PosPin Vtx Index" is just an index to the array stored internally as arbitrary data so the mesh and pins must be created somehow before setting these values.

I've found this thread where they try to do the same thing using scripting (also without success  )Create Puppet Pin on existing mesh via scripting

I have some ideas about how this could be done using SDK, but all of them involve diving in very dark waters..

- call the EffectSuite::AEGP_EffectCallGeneric method with fake mouse click events (PF_Cmd_EVENT) or some direct PF_Cmd_COMPLETELY_GENERAL commands if the FreePin effect supports it

- use "FreePin Suite" - which is not a part of public SDK but can be aquired using AquireSuite... - but as it's not public I suppose it operates with memory which is not so safe to touch even if I managed to get/create proper header file for this suite

Maybe there is a simple elegant way of creating these pins which I completely overlooked, so I would be really thankful for any suggestions

Thanks a lot!

Martin

[françois leroy]

Hi Martin,

I don't really see where you could find the 'FreePin Suite'... If you do, I'm REALLY interested in getting it too! I've spent days trying to do it the scripting way, and as you said, without success...

I think the 'Real' Mouse event is the only way to go. You can check this thread for info:

Any way to click a button on an effect?

When I say 'Real mouse event', I mean that it's probably easier to programatically perform this click and let the FreePin tool deal with everything it needs.

At least easier than trying to emulate this click 'inside' the FreePin and have to do all the hard work that's been already done on their side.

Now, I hope I'm wrong and we can access this suite directly...

Cheers,

François

[shachar carmi]

yo leroy! what's up?

if i'm not mistaking, the puppet tool adds some dynamic streams to the layer. perhaps these can be reverse engineered (sort of).

create a puppet effect with some pins in it. now recursively read that layer's streams, and read their match name.

now that you have that info, can these streams be created using the dynamic stream suite? (i don't know. i'm theorizing here)

all the stuff you need for this experiment is in the DynamicStreamSuite. (AEGP_AddStream, AEGP_GetMatchName, ect...)

[MartinMH]

yep, that's exactly what I've tried to do, but it doesn't work...

as I've said - when you create these streams manually (using dyn stream suite or via scripting) they are disabled (greyed out in UI) and the pins are not really present in the preview window...

btw. there is the "ADBE FreePin3 Outlines" stream of type AEGP_StreamType_NO_DATA called "Auto-traced Shapes"...

I'm not sure whether the NO_DATA streams can contain some arbitrary data (I think there is an AEGP_StreamType_ARB type for that purpose)

it would be easier to reverse engineer these streams if the arbitrary data was available via sdk api.. but apparently its not

moreover I don't really need to reverse engineer it completely - I just need to be able to serialize these streams and then create them on a different layer... something like copypasting it to a different layer... so if i could get a blob containing a raw binary data (if it was a simple structure containing only values not pointers etc) for these streams and if I was able to write this data blob to the new stream created using dyn streams suite, it would be enough for me...

all these values must be somehow serialized in aep file - so it should not be so hard to reverse engineer this structure, but it would be probably impossible to write them back without direct editing of aep file - which I really don't want to do... (reloading aep file would destroy undo stack etc)

anyway, thanks for suggestion!

[shachar carmi]

hmmm... digging through the project file sounds fishy, but it could work! i

mean, it's some arb data is involved (and i don't know if it is), then that

would be the way to go.

(and sorry for not having noticed you said you tried it already... it's yom

kippur <https://en.wikipedia.org/wiki/Yom_Kippur>, and all)

[françois leroy]

Hi Shachar! Long time I didn't bother you...

I'm not sure, the data is written into the file or not...

When you open a project, you can see in the info panel that FreePin is processing the mesh... It could mean some data is stored in the sequence data, but not flattened, and reprocessed on opening.

Also, when you reset the effect, you often get some pins greyed, like when you create them programatically. It's a bug, but could be a clue?

Just a thought though...

Cheers,

François

PS: have a good Yom Kippur!

[MartinMH]

françois leroy wrote:

I'm not sure, the data is written into the file or not...

When you open a project, you can see in the info panel that FreePin is processing the mesh... It could mean some data is stored in the sequence data, but not flattened, and reprocessed on opening.

Francois, I think you're right..

If you place a symbolic breakpoint (when debugging in Xcode) to a "EffectMainExtra" call in FreePin library, you will see that there is a lot of communication with the FreePin effect when the project is being opened. I think I should read something about accessing effect sequence data from AEGP plugin...

[françois leroy]

MartinMH a écrit:

I'm afraid it would require a lot of UI redrawing and ugly coords computations...

If you want to programatically set pins, I guess they'll be related to some points (or at least coords) from your own plugin, right?

Starting from there, you could get screen coordinates of these points from within your own plugin, and then just emulate those clicks.

Or am I missing something? What exactly do you want to do?

Just curious, cos' I'm in the middle of writing my own Mesh distortion plugin like FreePin, but of course much much better 😉

Cheers,

François

[MartinMH]

Yep, exactly, but I wanted to do it in the background without user interaction - so the comp could be inactive/closed.

It will be something like an importer.

And if I wanted to simulate real clicks, the composition must have been active (shown in composition window). Or am I wrong?

PS: I can't wait to see your new plugin

[françois leroy]

I guess you're right and composition has to be active...

But your comp will be active at some time (at least render time), so I see 2 options:

1_you make your comp active (programatically), emulate your clicks, and return to inactive state.

2_you process all the data you need when you import, store the result (click coordinates) somewhere, and launch only the clicks as soon as the comp becomes active.

I think option 1 is the safest.

[MartinMH]

Yep, but all these "real" click solutions have some really nasty issues...

If I'm not missing something, I would need to know magnification, offset and other params set in composition window to be able to compute mouse click coordinates or at least to find the layer in the comp window..

Moreover, emulating mouse clicks is OS specific, which is not a big issue... but still..

I think the more elegant solution would be to force the effect to flatten its sequence data and save it (export procedure) and then apply a new effect and force it to unflatten from the stored flattened data (import procedure).

Now I'm trying to play with AEGP_EffectCallGeneric, but I'm still not sure if its possible to do that.. will see again any suggestions will be highly appreciated

[françois leroy]

Hey Martin,

I've just made a little test, and the good news is emulating a click on FreePin does work 🙂

The created pin is valid!

Here's the windows code:

mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0);

mouse_event(MOUSEEVENTF_LEFTUP, 0, 0, 0, 0);

Pretty simple, isn't it? It clicks where the mouse is, now the hard work will be to set mouse position...

Cheers,

François

[MartinMH]

Thank you very much Francois! I really appreciate your help

As you've said the hard work is to find the correct position where to click..

It should not be so hard to find a comp window using os api calls, but even when you get the window coords you still need to find the layer there... which can be a bit tricky

Cheers,

Martin

[MartinMH]

Thanks Shachar, I'll try to investigate it a bit deeper...
Have an easy fast!

[shachar carmi]

thanks!

though the fasting has ended 2 hours ago, and i'm an atheist, so i wasn't

doing much fasting...

[MartinMH]

Hi François,

Thanks for suggestion!

Yep, that's another possibility, but I'm afraid it would require a lot of UI redrawing and ugly coords computations...

That's not really the way I'd like to go.. But as you've said, it's probably the only one that could really work.. Unfortunately...

"FreePin Suite" is used in Puppet effect plugin - I just discovered it by looking at the static disassembly of that binary, but I haven't investigated it deeper...

I've tried to call AquireSuite with this string and version 1 and it worked - by worked I mean it returned a pointer (to unknown structure as the headers for this suite are not public) and no error...

Cheers

Martin

[MartinMH · Accepted solution]

Voila, I've just found a workaround...

step 1: save an animation preset with one Puppet effect applied (with desired number of pins - but it doesn't matter where you place the pins)

step 2: apply this preset on a layer (using AEGP_ExecuteScript("app.project.item(index).layer(index).applyPreset(presetName);") ) - which will add the puppet effect with pins

step 3: modify vertex offset stream values of existing pins using stream suite

step 4: reset effect (which will reload the internal state - pins will jump to correct place)

step 5: modify vertex position streams using stream suite

So all you need is a set of .ffx files with different number of puppet pins used - because they cannot be added programatically - but it is possible to modify the existing ones...

Thanks a lot for your support!

Cheers,

Martin

[françois leroy]

Nice and smart!

Glad you found a way.

Cheers,

François

[cb43551664]

Martin,

I am trying to place and animate pins programmatically. My hope is to add 50-100 pins and set keys for each frame. Ideally I would do this with javascript but other forums indicate this is not possible. So I will go the way of the SDK.

This post seems to give the clues I need but I am stuck on this step.

3. modify vertex offset stream values of existing pins using stream suite

I am not familiar with this part of the API. Is there any good example code around? I will start with the example Streamie.h unless you have advice on code to look at.

Thank you,

Chris

[MartinMH]

Hi Chris,

Yep, you will need to find the stream/property you want to modify using combination of calls  AEGP_GetNewStreamRefByMatchname and AEGP_GetNewStreamRefByIndex (which is demonstrated in Streamie) and then set its value using AEGP_SetStreamValue. If the property is keyframed you will need to use KeyframeSuite for which I would recommend to look at Mangler, Easy_Cheese and ProjDumper examples.

Cheers,

Martin

[samig59966041]

Hi Martin,

I tried to follow your solution here but I am stuck at step 4. How do you reset the puppet tool effect with SDK?

When I do the reset manually in the UI, everything seems to work correctly. I am pretty new to SDK. Any help would be greatly appreciated.

Best,

Sami

[cb43551664]

Thanks, I will give it a try.

Chris