SmartFX: complete Full-resolution upstream input during reduced preview, including negative-origin text and continuously rasterized AI

Hi Shachar, thanks for taking the time to help me with this.

The low-alpha issue is about cell timing, not RGB values in fully transparent pixels. My effect divides the current upstream image into cells. At Full resolution, a pixel with alpha 4/255 makes one cell occupied. At Quarter resolution, that pixel disappears from the SmartFX checkout. The occupied-cell list then changes, which also changes the animation timing of other, fully visible cells.

I need the same current-frame upstream alpha information when AE requests Full or Quarter, including effects before mine on the same layer. I am not trying to retrieve an earlier frame.

I tried AEGP_NewFromUpstreamOfEffect with a Full-resolution render. For negative-origin editable text, ordinary SmartFX checkout at Full sees 21,436 visible pixels, but that auxiliary render sees only 13,598; it clips the negative-origin part.

Is there a supported way to obtain the complete Full-resolution result immediately before my effect during a Quarter-resolution SmartFX render? If not, can you confirm that this is a limitation of the API, so I can make an informed design decision?

Thanks again.

Thanks a lot, Shachar, for taking the time to help. I think I failed to explain what the effect actually does, which made the technical question confusing.

The effect receives the rendered RGBA image of its own layer and divides that image into a grid of rectangular cells. Each cell carries the pixels from its original part of the image and is animated independently.

Before creating the animation schedule, the effect checks which grid cells contain at least one nonzero-alpha pixel. Only those occupied cells participate in the timing order. This means that if reduced-resolution sampling makes a faint feature disappear, one cell can be removed from the timing order and the timing of the other, fully opaque cells can change.

I do not need the low-resolution output to contain every Full-resolution detail. I only need a separate Full-resolution image for this cell analysis. The actual preview can still be rendered at Half or Quarter resolution.

During Smart Render, on the same layer that contains the effect, I tried:

1. AEGP_NewFromUpstreamOfEffect
2. AEGP_SetDownsampleFactor(1,1)
3. AEGP_RenderAndCheckoutLayerFrame

In a simple PNG test, I placed the native Fill effect immediately before my effect. Changing Fill from red to blue also changes the Full-resolution image returned by this auxiliary render. At Full, that image matches the ordinary SmartFX input. It also remains Full resolution when the main render is at Half.

So this particular route does include the preceding Fill effect in my test. I have not yet proved the same thing for masks.

The remaining problem is the image extent. For the same editable-text frame at Full:

1. Normal SmartFX gives the region [-178,27,343,100].
2. The auxiliary upstream render gives [0,27,343,100].
3. The auxiliary image therefore loses the visible text at negative X coordinates.

By “complete,” I mean the same upstream pixels and extent that normal SmartFX supplies immediately before the effect. I am not asking AE to recover pixels removed by a mask.

The alpha example is also about nonzero alpha, not RGB beneath zero alpha. My 8bpc test image contains one pixel with alpha 4/255. With the effect disabled, that pixel is present in the Full export and absent in the Quarter export. That sampling difference is understandable. The problem is that analyzing the Quarter image removes one occupied cell and changes the animation timing of the opaque cells.

So my precise questions are:

1. Can the AEGP_NewFromUpstreamOfEffect route return the same complete extent as normal SmartFX, including negative-origin text?
2. If so, how should that extent or ROI be requested?
3. Is this render-and-checkout chain supported from Smart Render and under MFR?
4. Should previous masks also be present in this upstream-of-effect image?

Thanks again for your patience. Your replies are helping me determine whether I am using the API incorrectly or have reached a limitation of this rendering route.

1. the only way to obtain full res images during downsampled render calls is exactly the way you did it. however, it will not return the result of upstream effects, but rather in source layer’s buffer. you can receive the source image at full res and uncropped at most. what else are you hoping to get? the source image can get cropped/expanded/offset by masks and effect on the same layer, but as mentioned, you can’t get the rendered image to inculde these processes using AEGP_RenderAndCheckoutLayerFrame, and the smart render layer chekcout which can, won’t give you any resolution other than the currently rendered downsample settings.
   
   as for the text layers getting cropped, when out of sight, that is actually the expected behavior. text layers are passed post transform as any “collapsed transformation” layer is. so when the text is out of the comp’s bounds, it’s not hidden, it just doesn’t exist, as if it were out of bounds in some internal comp. did i understant the situation correctly? if not, please elaborate.
    
2. the aux layer (a layer selection param, if i understanf the setup correctly?) behaves in the same way is the input layer when checked via AEGP_RenderAndCheckoutLayerFrame. so you should expect the same behaviour as when checking the effect’s main input. is that not what you’re getting?
    
3. mfr participance is declared in the effect’s outflags on global setup. after that, the effect just gets called to render at random times for random frames (i.e. not sequentially). there is no pre-process besides the pre-render call. from the plugin’s perspective, mfr and non-mfr renders look the same.
   
   the one difference is the sequence data handling, which may differ when telling AE an MFR plugin needs separate sequence data chunks for each rendering thread. but that is done separately of the render calls.