Expanding the buffer via SmartFX.

[dkt0]

I am getting completely ignored when it comes to buffer expansion via SmartFX and I do not know why. Here is what I learned until now:

- Both rects must be set to the expanded rect

- Account for buffer expansion in SmartRender

Here is the issue I currently have (expanding the buffer 200 pixels in each direction):

The buffer only expands in the top-left part and it looks smth like this:

It should clearly look something like this instead:

Now, absolutely no one wants to help, I don't have the slighest idea why this happens. The buffer expands in the top-left direction but it shrinks in the bottom-left direction. It does not make any sense. Here is my current code (btw if you set both rects to the expanded rect, PF_RenderOutputFlag_RETURNS_EXTRA_PIXELS is not needed:

static PF_Err
PreRender(
	PF_InData* in_data,
	PF_OutData* out_data,
	PF_PreRenderExtra* extra)
{
	PF_Err err = PF_Err_NONE;
	PF_RenderRequest req = extra->input->output_request;
	PF_CheckoutResult in_result;

	ERR(extra->cb->checkout_layer(in_data->effect_ref,
		SKELETON_INPUT,
		SKELETON_INPUT,
		&req,
		in_data->current_time,
		in_data->time_step,
		in_data->time_scale,
		&in_result));


	PF_LRect expanded_rect = in_result.result_rect;
	expanded_rect.left -= 200;
	expanded_rect.top -= 200;
	expanded_rect.right += 200;
	expanded_rect.bottom += 200;

	extra->output->result_rect = expanded_rect;
	extra->output->max_result_rect = expanded_rect;

	extra->output->flags |= PF_RenderOutputFlag_RETURNS_EXTRA_PIXELS;

	return err;
}


static PF_Err
SmartRender(
	PF_InData* in_data,
	PF_OutData* out_data,
	PF_SmartRenderExtra* extra)
{
	PF_Err err = PF_Err_NONE;
	AEGP_SuiteHandler suites(in_data->pica_basicP);

	PF_EffectWorld* input_worldP = NULL;
	PF_EffectWorld* output_worldP = NULL;

	ERR(extra->cb->checkout_layer_pixels(in_data->effect_ref, SKELETON_INPUT, &input_worldP));

	if (!err) {
		ERR(extra->cb->checkout_output(in_data->effect_ref, &output_worldP));

		if (!err && input_worldP && output_worldP) {
			PF_ParamDef x_param, y_param;
			AEFX_CLR_STRUCT(x_param);
			AEFX_CLR_STRUCT(y_param);

			ERR(PF_CHECKOUT_PARAM(in_data, SKELETON_X, in_data->current_time, in_data->time_step, in_data->time_scale, &x_param));
			ERR(PF_CHECKOUT_PARAM(in_data, SKELETON_Y, in_data->current_time, in_data->time_step, in_data->time_scale, &y_param));

			if (!err) {
				A_long origin_x_offset = input_worldP->origin_x - output_worldP->origin_x;
				A_long origin_y_offset = input_worldP->origin_y - output_worldP->origin_y;

				OffsetInfo offsetInfo;
				offsetInfo.xOffset = x_param.u.fs_d.value + origin_x_offset;
				offsetInfo.yOffset = y_param.u.fs_d.value + origin_y_offset;
				offsetInfo.input_world = input_worldP;

				PF_PixelFormat pixelFormat;
				PF_WorldSuite2* wsP = NULL;
				ERR(suites.Pica()->AcquireSuite(kPFWorldSuite, kPFWorldSuiteVersion2, (const void**)&wsP));
				ERR(wsP->PF_GetPixelFormat(output_worldP, &pixelFormat));
				ERR(suites.Pica()->ReleaseSuite(kPFWorldSuite, kPFWorldSuiteVersion2));

				switch (pixelFormat) {
				case PF_PixelFormat_ARGB128:
					ERR(suites.IterateFloatSuite1()->iterate(
						in_data,
						0,
						output_worldP->height,
						input_worldP,
						NULL,
						(void*)&offsetInfo,
						(PF_IteratePixelFloatFunc)MyOffsetFunc<PF_PixelFloat>,
						output_worldP));
					break;

				case PF_PixelFormat_ARGB64:
					ERR(suites.Iterate16Suite1()->iterate(
						in_data,
						0,
						output_worldP->height,
						input_worldP,
						NULL,
						(void*)&offsetInfo,
						MyOffsetFunc<PF_Pixel16>,
						output_worldP));
					break;

				case PF_PixelFormat_ARGB32:
				default:
					ERR(suites.Iterate8Suite1()->iterate(
						in_data,
						0,
						output_worldP->height,
						input_worldP,
						NULL,
						(void*)&offsetInfo,
						MyOffsetFunc<PF_Pixel8>,
						output_worldP));
					break;
				}

				PF_CHECKIN_PARAM(in_data, &x_param);
				PF_CHECKIN_PARAM(in_data, &y_param);
			}
		}
	}

	if (input_worldP) {
		ERR(extra->cb->checkin_layer_pixels(in_data->effect_ref, SKELETON_INPUT));
	}

	return err;
}

And also some debugging with default values (0/0):

[2025-06-15 17:18:43] === PreRender START ===
[2025-06-15 17:18:43] Original request rect: left=-108, top=-135, right=1188, bottom=1485
[2025-06-15 17:18:43] Input result rect: left=0, top=0, right=1080, bottom=1350
[2025-06-15 17:18:43] Before expansion: left=0, top=0, right=1080, bottom=1350
[2025-06-15 17:18:43] After expansion: left=-200, top=-200, right=1280, bottom=1550
[2025-06-15 17:18:43] === PreRender END ===

[2025-06-15 17:18:43] === SmartRender START ===
[2025-06-15 17:18:43] Input world: width=1080, height=1350, origin_x=0, origin_y=0
[2025-06-15 17:18:43] Output world: width=1480, height=1750, origin_x=-200, origin_y=-200
[2025-06-15 17:18:43] Origin offsets: x_offset=200, y_offset=200
[2025-06-15 17:18:43] Parameters: x_param=0, y_param=0
[2025-06-15 17:18:43] Final offsets: xOffset=200, yOffset=200
[2025-06-15 17:18:43] Buffer bounds check: Input bounds: (0,0) to (1079,1349) Output bounds: (0,0) to (1479,1749)
[2025-06-15 17:18:43] Expansion analysis: Left expansion: YES Top expansion: YES Right expansion: YES Bottom expansion: YES
[2025-06-15 17:18:43] === SmartRender END ===

[shachar carmi · Accepted solution]

copying my answer here as well:

so...

i put together a working sample the shows buffer resizing in smartFX. (based on the "smarty pants" sample)

download it from here:
https://drive.google.com/file/d/1w4Fc9rtGXjU-YT4CC-6q01ft2MOvOLtt/view?usp=sharing

from what i gathered during the making of this was that you should put the expanded buffer rect in "extra->output->result_rect".

seems AE will take just that rect. i didn't see AE asking for the max_result_rect yet...

so depending on your implementation, you should decide *what* get's expanded.

is it relative to any buffer you get, be it cropped by a mask or the comp bounds?
is it relative to the original layer size?
is it relative to the max rect of the input?

all these very much depend on your implementation.

in the sample project, i expanded the input buffer whatever it may be, and put it in the "extra->output->result_rect". then i expanded the "in_result.max_result_rect" and put the expanded result in "

extra->output->max_result_rect".

from all i've seen, AE always used the rect from "extra->output->result_rect" for the render call, even when the layer was scaled down and exposed the area around the original layer that it *could* grow into.

[dkt0]

The only difference I can see is the values used for the expansion, mine uses hardcoded values (200) while your depends on the parameters. I modified my Directional Blur plugin to use the same approach, it just does not work, it's completely different, yours just fills the buffer with a color, I've actually managed to make a plugin myself that does that. What do you do with a blur plugin or a plugin that moves the layer? Look through my plugin's code when you have time:

https://github.com/NotYetEasy/AE-Motion/tree/main/DirectionalBlur

I have a repository with all my plugins, but I only modified the Directional Blur to have buffer expansion. I'm honestly so tired of looking for the problem, been doing it for weeks now. I just can't pinpoint it. This way you can get the plugin, compile it and see for yourself what issues I got. I really wanna move on from this buffer expansion thing.

[shachar carmi]

i modified the previous project to do a gaussian blur that expands the buffer as the blur size grows.

you can see how the original image is centered in the intermediate buffer that matches the expanded output.

here's the new version:

https://drive.google.com/file/d/1JWtvacj-jt5I8XfqxCzz3k_8lGi_nLaI/view?usp=sharing

[dkt0]

Actually, your first plugin was helpful too, took me a while to understand exactly how it worked, but I managed to fix my plugins too, working on them right now. You have an interesting approach for this, lol 🙂

Btw, pre_effect_source_origin_x/y does not seem to work with your approach, apparently we can account for that using smth like this:

int actual_expand_x = (output_worldP->width - input_worldP->width) / 2;
int actual_expand_y = (output_worldP->height - input_worldP->height) / 2;

Anyways, thanks so much man, this was prolly my last obstacle. You helped me so much and I thank you A LOT for that!

[dkt0]

Btw, is there any way to access the letters of a text layer? Like idk, make an effect that spaces out the letters? (I know you can already do that in AE, but it would be the easiest plugin to make that works with text layers).

[dkt0]

There isn't any actual code here...