Writing a Plugin that scales the layer applied to

[Mallowed]

Hi, im new to writing ae plugins and wanted to try figure out how to write a simple plugin that can scale the footage,layer etc... of what its applied to. 

i understand in logic of how to do it:

find out layer w&h in pixels

CurrentscaleX = 100;

CurrentScaleY = 100;

scaleP.ScaleMultipler = params[EXAMPLE_SCALE]->u.fs_d.value / 100;     ie 101/100= 1.01 scale increase  

if (ScaleP ) {

 NewscaleX = CurrentScaleX * Scale Multiplier; 

 NewscaleY = CurrentScaleY * Scale Multiplier; 

CurrentScaleX = NewScaleX;

CurrentScaleY = NewScaleY;

}

but as to writing that logic into a format the sdk can understand i have no idea. would be great if i could get a bit of help.

[Mylenium]

Welcome to the Adobe After Effects SDK Guide! — After Effects SDK Guide 22.0.0 documentation

Mylenium

[shachar carmi]

generally speaking, a plug-in receives an input buffer with pixels to mnipulate, and an output buffer to put the results into. *how* the output buffer is filled is entirely up to you. AE doesn't care.

so to scale your image you could use your won algorithm to shuffle pixels around, use 3rd party libraries, OR use some of the tools supplied by AE's API.

for example, you could use transform_world() from PF_WorldTransformSuite1 to transpose, scale to rotate an image.

here's a thread talking aobut it:
https://community.adobe.com/t5/after-effects-discussions/help-with-transform-world/m-p/1840909

search the forum for more info, this function has been talked about here on numerous occasions.

[Mallowed]

thank you for this, i saw in the sdk documentation about transform_world() but didnt know how to execute it. do just code it directly into the render section of the cpp file or do i use a err(suites.pf_worldtransfomsuite1 )-> and pass it through to a func8?

sorry if this seems like a silly question but still very new to this 

[shachar carmi]

yes, the process you described is correct. take a peek at the sample projects in the SDK, they all (the effects at least) show a similar concept for handling the render.

now i'll go a bit deeper.

the only thing that really matters in regards to rendering, is that the ouptut buffer is returned filled within the scope of the render call. that is, you can't tell AE "i'll return now, bu t pass the result later". once your plug-in returns from a render call, AE will use the ouput buffer for downstream processes.

the rendering code itself doesn't have to reside entirely in the render function. you can write classes to handle that, you can send the innputs to a remote server for processing (don't do that. i'm just being rehtorical), AE doesn't know or care as long as the output buffer is filled by the time your plug-in returns from the render call.

some stuff to consider:

to use AE's suites, you would (probably) want to use the suite handler (otherwise it's a hassle to aquire and release the suites). to use that handler you need to pass it the basic_picaP pointer from in the in_data structure.

now we're getting to the point. that in_data structure is only valid within the scope of a single AE call to the plugin. if you try to store the in_data and use it after the call in which that data was sent has returned, that data is now garbage and using it will cause your plug-in to crash.

that being said, the basic_picaP pointer itself is consistent throughout the AE session, and is safe to store.

[Mallowed]

thank you that is very usefull information but what im struggling with is the data that world_transfrom() requires,

here is what i have so far but is returning a error at the ERR():

static PF_Err

Scale8(

void* refcon,

A_long xL,

A_long yL,

PF_Pixel8* inP,

PF_Pixel8* outP)

{

PF_Err err = PF_Err_NONE;

ScaleInfo *ScaleP = reinterpret_cast<ScaleInfo*>(refcon);

if (ScaleP) {

ScaleP->ScaleFactor;

ScaleP->Amplitude;

ScaleP->translateX;

ScaleP->translateY;

}

return err;

}

static PF_Err 

Render (

PF_InData *in_data,

PF_OutData *out_data,

PF_ParamDef *params[],

PF_LayerDef *output )

{

PF_Err err = PF_Err_NONE;

AEGP_SuiteHandler suites(in_data->pica_basicP);

/* SCALEFUNCT. */

ScaleInfo ScaleP;

AEFX_CLR_STRUCT(ScaleP); // clears the struct

A_long scaleX = 1;

A_long scaleY = 1;

A_long skewX = 0;

A_long skewY = 0;

A_long posX = 0;

A_long posY = 0;

A_long rotationAngle = ScaleP.Amplitude * (M_PI / 180);

PF_CompositeMode mode;

mode.opacity = 255;

mode.opacitySu = 32768;

mode.rgb_only = false;

mode.xfer = PF_Xfer_IN_FRONT;

Matrix2D transformationmatrix;

transformationmatrix.a = scaleX * ScaleP.ScaleFactor * cos(rotationAngle);

transformationmatrix.b = skewY * sin(rotationAngle);

transformationmatrix.c = skewX * cos(rotationAngle);

transformationmatrix.d = scaleY * ScaleP.ScaleFactor * sin(rotationAngle);

transformationmatrix.tx = posX + ScaleP.translateX;

transformationmatrix.ty = posY + ScaleP.translateY;

PF_Rect dest;

dest.bottom = in_data->height;

dest.left = 0;

dest.right = in_data->width;

dest.top = 0;

int quality = PF_Quality_HI;

ScaleP.ScaleFactor = params[MSHAKE_SCALE]->u.fs_d.value;

ERR(suites.WorldTransformSuite1()->transform_world( in_data,

quality, // scaling quality? LO/HI = blinear/detail presevering upscale??

0, // probably set to 0?? (ik this is flags and its fine to set to 0 but still dont understand what it does)

PF_Field_FRAME, // field (still not sure what this does)

&params[MSHAKE_INPUT]->u.ld, // src_world (still not sure what this does)

&mode, // comp_mode (not sure what this does)

NULL, // null (no mask data so this takes the whole frame)

&transformationMatrix, // define m - 1 is &m 2 is (&m1, &m2)

1, // matrices amount 1 is transfrom 2 is motion blur and transfrom

true, //src2dts_matrix (still not sure what this does)

&dest, //dest_rect - calc the whole screen

Scale8)); // output (do i pass to my func here?)

return err;

}

[shachar carmi]

ooohhh... that's a bit of a mess.

first off, the docs are wrong about putting in_data on the first arg. the best way to figure out how to use a callback (if you don't find it in the SDK sample projects) is to go to the function definition in the code. here's what it looks like there:

PF_Err (*transform_world)(
PF_ProgPtr effect_ref,
PF_Quality quality,
PF_ModeFlags m_flags,
PF_Field field,
const PF_EffectWorld *src_world,
const PF_CompositeMode *comp_mode,
const PF_MaskWorld *mask_world0,
const PF_FloatMatrix *matrices,
A_long num_matrices,
PF_Boolean src2dst_matrix,
const PF_Rect *dest_rect,
PF_EffectWorld *dst_world);

so now we can see the first arg is actually the effect_ref (in_data->effect_ref) that should go there, but i put a NULL instead. (otherwise it checks for interrupts)

in the last arg, you need to put a PF_EffectWorld, not a function. the output buffer would do. transform_world does the calculations of what pixel goes where according to the passed matrix.

for a matrix you need to fill a PF_FloatMatrix. read up on transformation matrices. just note that AE's matrices are column based instead of row based as is usually demostrated, so you might need to change the pocation of the elements from x,y to y,x. (it's called transposing a matrix)

about PF_Field_FRAME. frames in videos might be interlaced (where even rows come from one moment in time and odd rows from another). in the in_data, ae tells you is the input is interlaced or not, and you should act accordingly throught your process. if all you do is change pixel colors, then it doesn't matter. if you're doing scaling it matter a lot, because pixels originating from odd lines must end up on odd lines or they will appear on screen at the wrong time. read up on interlacing.

src2dst_matrix tells AE that the passed matrix transfroms points from source coordinates to destination coordinates. if set to false, the transformation will be reversed. 

PF_CompositeMode tells AE how to treat the incoming pixels in regards to pixels already inside the dst buffer. so it doesn't only transform but only composites the transformed image on top of the existing buffer. hence, the opacity and transfer mode options.

[Mallowed]

thank you for writing this it helps alot understanding what stuff are needed to pass through the transform_world()

how would i pass my function through it though? 

would you be able to help me structure my func8 and how i need to set up my render from my code above?

i dont know if i need the PF_err(world_transform) in my render or i can do it in the func8 and then just call for it when rendering? 

[Mallowed]

i feel like i should give context as to what i am trying to do,

i created this expression within after effects:

var rotRight = effect("Right Amplitude (A)")("Slider");
var rotLeft = effect("Left Amplitude (B)")("Slider");
var shakeDirection;
var scaleClamp = effect("Detail-preserving Upscale")("Scale");
var minScale = effect("Scale Min Clamp ")("Slider");
var maxScale = effect("Scale Max Clamp")("Slider");
var nonClamp = effect("Amplitude")("Slider");
if (effect("Shake Logic")("Menu").value ==1) {
if (effect("Shake Direction Bias")("Menu").value ==1) {
var seedGen = effect("Seed")("Slider");
seedRandom(seedGen, true);
var shakeBias = random();
if (shakeBias < 0.5) {
shakeDirection = rotLeft;
} else {
shakeDirection = rotRight;
}
} else if (effect("Shake Direction Bias")("Menu").value ==2) {
shakeDirection = rotLeft;
} else if (effect("Shake Direction Bias")("Menu").value ==3) {
shakeDirection = rotRight;
}

// shake logic calc

if (effect("Clamped to Scale?")("Checkbox") == true) {
var LinearRotation = linear(scaleClamp, minScale, maxScale, 0, shakeDirection);
var noiseRotation = noise(time + scaleClamp * (effect("Frequency")("Slider")));
var easedRotation = ease(scaleClamp, minScale, maxScale, 0, 1);
var finalRotation = LinearRotation * noiseRotation * easedRotation;
rotation = finalRotation*effect("Dissolve Multiplier")("Slider");
} else {
var LinearRotation = linear(nonClamp, minScale, maxScale, 0, shakeDirection);
var noiseRotation = noise(time + scaleClamp * (effect("Frequency")("Slider")));
var easedRotation = ease(nonClamp, minScale, maxScale, 0, 1);
var finalRotation = LinearRotation * noiseRotation * easedRotation;
rotation = finalRotation*effect("Dissolve Multiplier")("Slider");
}

} else {
// redundant code because expressions have limitations
var seedGen = effect("Seed")("Slider");
seedRandom(seedGen, true);
var shakeBias;
if (shakeBias < 0.5) {
shakeDirection = rotLeft;
} else {
shakeDirection = rotRight;
}
var sinwaveFrequency;
var sinwaveAmp;
if (effect("Use A/B values instead")("Checkbox") == true) {
sinwaveFrequency = effect("Frequency")("Slider");
sinwaveAmp = shakeDirection;
} else {
sinwaveFrequency = effect("Sin Wave Frequency")("Slider");
sinwaveAmp = effect("Sin Wave Tilt Amplitude")("Slider");
}
shakeDirection = Math.sin((time-effect("seed")("slider"))*effect("Sin Wave Frequency")("slider"))*sinwaveAmp*effect("Amplitude")("Slider")

if (effect("Clamped to Scale?")("Checkbox") == true) {
var LinearRotation = linear(scaleClamp, minScale, maxScale, 0, sinwaveAmp);
var noiseRotation = noise(time + scaleClamp * sinwaveFrequency);
var easedRotation = ease(scaleClamp, minScale, maxScale, 0, 1);
var finalRotation = LinearRotation * noiseRotation * easedRotation;
rotation = finalRotation*effect("Dissolve Multiplier")("Slider");
} else {
var LinearRotation = linear(nonClamp, minScale, maxScale, 0, sinwaveAmp);
var noiseRotation = noise(time + scaleClamp * sinwaveFrequency);
var easedRotation = ease(nonClamp, minScale, maxScale, 0, 1);
var finalRotation = LinearRotation * noiseRotation * easedRotation;
rotation = finalRotation*effect("Dissolve Multiplier")("Slider");
}
}

but i wanted to see if i could write it as plugin as i wanted to add more to it and wanted to clean it up as it looks horridly messy with not being able to have drop down sub menus to tidy elements

[shachar carmi]

i see.

let's tell a few things apart:

1. the means of determining what the transformation should be like at a given time.

2. putting all the transformation info into a matrix.

3. transforming the input image with said matrix.

so:

1. that's pretty much a translation of your expression to c++, ending up with x/y position, rotation and x/y scaling values for a given time. i trust you don't have an issue there.

2. why do you need a matrix? because rendering a transformation is no trivial manner which requires a TON different algorithms for sampling the image when upsclaed and downscaled, and a TON of optimizations to run it fast. luckily, the AE api offers you transform_world which accepts a matrix and does the rest. that's why you need a matrix. you need to read up on transformation matrices and how to construct them. but for testing sake, here's how to fill the PF_FloatMatrix to downscale the image by half:

0.5, 0, 0,

0, 0.5, 0,

0, 0, 1

matrices are kind of magical. you can concatenate multiple transformations into one matrix, and when multiplying an input point's x/y coordinates (a vector) by the matrix, you magically get the x/y coordinates of where it should appear on the output buffer.

3. now that you have a matrix contaning all of your transformation decisions put together, you can call transform_world with an input buffer to take pixels from, output buffer to put pixels to, and the matirx with all your transformation. transform_world does not talk to any other functions. all it does is move the pixels from the input to the output according to the matrix.

[Mallowed]

yeah i looked into the float matrix and saw how mine was incorrect. this is what i have now. its still incorrect but runs without error (the color of the footage just messes up and nothing else happens i think its because its pushing values past 255? which i can fix )

here is my code now:

Scale8(

void* refcon,

PF_InData* in_data,

PF_OutData* out_data,

PF_ParamDef* params[],

PF_LayerDef* output,

PF_Pixel8* inP,

PF_Pixel8* outP)

{

PF_Err err = PF_Err_NONE;

AEGP_SuiteHandler suites(in_data->pica_basicP);

ScaleInfo *ScaleP = reinterpret_cast<ScaleInfo*>(refcon);

/*variables*/

PF_Rect dest;

dest.bottom = in_data->height;

dest.left = 0;

dest.right = in_data->width;

dest.top = 0;

int quality = PF_Quality_HI;

PF_CompositeMode mode;

mode.opacity = 255; // 8bit max value

mode.opacitySu = 32768; //for deep color ie. 16 and 32bit

mode.rgb_only = false; //data type is rgba

mode.xfer = PF_Xfer_IN_FRONT; // renders on top

/*defined values*/

PF_FpLong rotationAmount = ScaleP->Amplitude * (M_PI / 180);

PF_FpLong MultiplierAmount = ScaleP->ScaleFactor / 100;

/*matrix*/

PF_FloatMatrix m;

void setuptransformmatrix(PF_FloatMatrix* m, float posX, float posY, float rotationAngle, float scaleX, float scaleY);

for (int i = 0; i < 3; i++) {

for (int j = 0; j < 3; j++) {

if (i == j) {

m.mat[i][j] = 1.0f;

}

else {

m.mat[i][j] = 0.0f;

}

}

}

m.mat[0][2] = 0 + ScaleP->translateX;

m.mat[1][2] = 0 + ScaleP->translateY;

m.mat[0][0] = cos(rotationAmount); //rotation along x axis

m.mat[0][1] = -cos(rotationAmount);

m.mat[1][0] = sin(rotationAmount); // Rotation along Y-axis

m.mat[1][1] = cos(rotationAmount);

m.mat[0][0] = 1 * MultiplierAmount; // scaleX

m.mat[1][1] = 1 * MultiplierAmount; // scaleY

PF_Err(suites.WorldTransformSuite1()->transform_world(

NULL, //PF_ProgPtr effect_ref,

quality, //PF_Quality quality,

0, //checks for flags,

PF_Field_FRAME, //checks for interlacing??,

&params[MSHAKE_INPUT]->u.ld, //uses the params for the input id,

&mode,

NULL,

&m, 

1,

true,

&dest,

output /*i know im supposed to use the PF_EffectWorld but dont know how to call on it*/ ));

if (ScaleP) {

ScaleP->ScaleFactor;

ScaleP->Amplitude;

ScaleP->translateX;

ScaleP->translateY;

}

return err;

}

static PF_Err 

Render (

PF_InData *in_data,

PF_OutData *out_data,

PF_ParamDef *params[],

PF_LayerDef *output )

{

PF_Err err = PF_Err_NONE;

AEGP_SuiteHandler suites(in_data->pica_basicP);

/* SCALEFUNCT. */

ScaleInfo ScaleP;

AEFX_CLR_STRUCT(ScaleP); // clears the struct

ScaleP.ScaleFactor = params[MSHAKE_SCALE]->u.fs_d.value;

return err;

}

[Mallowed]

i was just reading the CCU and do i need a seperate func for each rot, scale and pos matrices if i want to adjust them seperate to each other, so they would all have their own individual transform_world?

[shachar carmi]

oh gawd no! construct one matrix and do the transormation only once. both for performance sake and for not re-sampling the image 3 times which will blur your image in a nasty way.

you're putting the correct stuff in the correct places in the matrix, but these operations need to be multiplied as whole matrices for each of the transformation operations, and not multiplying the separate components in the matrix.

lookup matrix multiplication. you'll find a function in no time.

but here's some shortcuts before going on this adventure:

the default matirx which does nothing is rederred to as "identity mtrix". it's filled as follows:

1, 0, 0,

0, 1, 0,

0, 0, 1,

use that as the base for each of your separate transform, scale and rotate matrices, and then multiply them.

2. matrix operations happen relative to the top left corner of the image. if you want it relative to the center you need to first create an transpose matrix for half the width and height.

so what you'd actually want to do to construct a proper transformation matrix is:

1. create a transpose matrix for negative the anchor point value. that moves the image's pivot point to the matrix's 0,0 coordinate.

2. multiply that matrix by the scale matrix, which will now happen around the anchor point.

3. multiply the previous result by the rotation marix (again, will happen aroung the expected point)

4. multiply the previous result by another transform matrix with the x/y position values.

that's it. your matrix is ready to use.

when going more advanced, you'll want to factor in pixel aspext ration as well. it would just mean doing a scaling matrix on x and the begining and end steps.

[Mallowed · Accepted solution]

I cannot thank you enough for all the help through this. i have the image scaling through the slider now without issue, i need to make it scale from the center of the layer but im sure i can figure out the rest myself, i cannot thank you enough through all of this. you are a legend,

Mal

[shachar carmi]

i'm marking your answer as correct because it contains only true statements.