AI objekt removal tool

With Photoshop or Lightroom, removing unwanted objects in your frame is a breeze nowadays. I’m suggesting a similar feature for After Effects. Thank you, Rebecca, for taking notes for this idea at IBC26.

I imagine it working like this:

The first step is to create a mask, either manually or with the same engine as the new object mask tool in Premiere. Then, within a new floating window (as in Photoshop), you can either click to remove the object or describe a prompt for what to add. Also, there should be an option for a reference image (with an eyedropper to quickly grab a frame from the project). Maybe you can also reference a composition. There also should be an option to choose the right video model. Maybe also a button to expand/shrink the mask when the selection is very tight. This would avoid hard edges between an object to remove and the background, and the AI would give more room to work with.

Removing object:
When removing objects, the AI should take temporal information into account. For example, you have a shot of a person in front of a building, but a car is driving through the background, and you want to remove it. There are enough “good” frames of the house the AI can reference when trying to remove the car.

Also there should be the option for a prompt:
Maybe you have a person and want to give it a hat. You mask around the head and the room above and type your prompt. In this spot the AI will create footage of a hat there.

And the last idea is referencing a comp:
Example: You mask around a part of the shirt of the person in the frame. You prompt, “Add the logo there," and reference a comp with a logo (maybe even animated). The comp has to have at least the same length as the main clip. Then the AI adds the content of the comp to the desired spot but also takes into account lighting, position, stretching, deformation, etc. It also takes the first frame of the comp as the first reference frame and so on. This means if you have a logo where it cycles through colors or the shape changes, the AI adds the logo to the shirt, and the color or the shape matches what you set up in the comp. I can imagine this would be an easy method for changing the content of phone or TV screens.

I hope the community agrees on these ideas and Adobe will listen. Especially the object removal would make the life of many editors easier. Also there is a similar idea in the premiere community but I think this is better suited for after effects: