my content aware fill always looks bad

[Ivan_Rizzo]

Since ever, I cannot get the content aware fill to have decent results, no matter how simple the scene is. And it confuses me when I see tutorials on youtube where people, usually, get decent fills with just a few clicks.

I'm attaching a video of my application of the content aware fill in a scene and another of the sample I used so you can see exactly what I'm doing  and what could be missing wrong.

[Rick Gerard]

Content-aware fill is the wrong tool for that job. The fastest workflow would be to track that shot in Mocha and Stabilize it so the cup doesn't move. Create a rectangular spline around the cup, track Transform, Scale, and Rotation only, increase the pixels used to about 75%, and track. 

After saving and returning to After Effects, Expand Tracking Data, Invert it, Retrieve the Cup Track from Mocha, Add a Null to the timeline, and apply only the transform data to the null. Then, Parent the Footage to the Null, and the cup will be stabilized.

From the first frame of the timeline, use the Composition menu to Save Frame As/Photoshop Layers, open the PSD in Photoshop, and use the Clone tool and fill to remove the Cup from the shot, then Create a Layer mask so only the area without the cup is showing, save the file, and import as footage and choose background masked layer - do not crop to layer size.

Drop the Imported PSD in the AE timeline, Remove the parenting of the footage from the Null, Uncheck Invert in Mocha/Tracking Data, and add the Cup track one more time to make sure the data is correct. After Verifying that the camera movement is back in the shot because you removed the parenting.

Parent the PSD layer to the Null now, and the replacement part of the image will now track with the rest of the shot. 

Content-Aware Fill needs a source for the pixels. If the coffee Cup were moving across the desk, there would be image details that appear behind the cup as it moves away. AI is not smart enough yet to know what it looks like behind the Coffee Cup. Photoshops' AI features can get you close, but you still need to do some cloning and hand painting to hide the cup. Motion tracking, or Stabilizing and putting the motion back in the replacement background, is the only efficient way to fix this shot. A car driving down a road can be removed with CAF, but a car parked on the street can't be removed by AI because it is incredibly difficult to accurately predict what is behind the car unless it moves away.

If there were a lot of perspective shifts in the shot, you would use a stabilized corner pin workflow to replace the elements in the shot that needed fixing. Here's a tutorial I produced on that workflow.

I hope this helps.