Highlighted

Remove background artifact?

Explorer ,
Sep 21, 2020

Copy link to clipboard

Copied

I'm making "vitrual choir" videos, so each video that I receive from a choir member is shot with the camera on a stable platform, with the singer in the foreground. Now, suppose the background contains some artifact that is never occluded by the foreground action, such as a dark item on a shelf in an otherwise light-colored background. I'm fishing for ways to get rid of the background artifact.

 

The way I do it today, which works but is a lot of work, is to hide the artifact with a matte as follows:

  1. Export a frame from the video
  2. Edit the video frame in Photoshop:
    1. Draw a marquee around the objectionable object
    2. Apply "Content-aware fill" to "remove" the object from the picture
    3. Invert the selection (ctrl-shift-I on Windows)
    4. Delete everything (to transparent) that isn't the "content-aware fill"
    5. Save the resultant image file. I call it the "matte".
  3. Import the image file and place the matte in the timeline above the original video. Just like magic, the object is gone (hidden by the mostly-transparent matte).
  4. But wait, there more... It turns out that the brightness of the surrounding area in the picture is probably not constant. So, as the video progresses, if the surrounding area in the video gets brighter or darker (because the camera is doing some sort of auto-brightness thing) then the matte appears to be a sharply-defined darker or brighter area against the image. So (and this is the time-consuming part):
    1. Select the matte layer
    2. Apply the Lumetri color effect
    3. Go to the beginning of the timeline, and enable keyframes on color wheels/mid-tone brightness.
    4. Switch to the Color workspace, and display the color wheels section
    5. Play (or manually scrub) the video. As it progresses, you will see the luma in the scope move up or down as the camera adjusts its brightness. At the same time, you will see your matte become visible against the background. Slide the mid-tone slider up or down to bring everything back into equilibrium.
    6. Repeat that process. Again. And again. And play it back and fix the places where you didn't get it quite right...

 

So... This works, but the brightness thing takes a lot of work to sort out. I asked a separate question and, apparently, there is no way to automate things so that the area around the non-transparent matte remains at a constant brightness.

 

Another option, which wouldn't be as seamless but might be "good enough" and a lot less work, would be to define a marquee around the objectionable object in the image and make all of the colors in that object match the surrounding color. But I haven't found a way to do that well enough that the area doesn't stand out worse than the original "objectional object" in the image.

 

Any suggestions?

Adobe Community Professional
Correct answer by John V Knowles - WM | Adobe Community Professional

There is a Content-Aware Fill option in After Effects now; if you're already familiar with the one in Photoshop then give the one in AE a try. You draw a maks around the object and it tries to fill in the blank. It may also try to replicate the lighting shifts in the footage, though there's no guarantee that it will work 100% perfectly. 


WM

TOPICS
Effects and Titles, How to

Views

64

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

Remove background artifact?

Explorer ,
Sep 21, 2020

Copy link to clipboard

Copied

I'm making "vitrual choir" videos, so each video that I receive from a choir member is shot with the camera on a stable platform, with the singer in the foreground. Now, suppose the background contains some artifact that is never occluded by the foreground action, such as a dark item on a shelf in an otherwise light-colored background. I'm fishing for ways to get rid of the background artifact.

 

The way I do it today, which works but is a lot of work, is to hide the artifact with a matte as follows:

  1. Export a frame from the video
  2. Edit the video frame in Photoshop:
    1. Draw a marquee around the objectionable object
    2. Apply "Content-aware fill" to "remove" the object from the picture
    3. Invert the selection (ctrl-shift-I on Windows)
    4. Delete everything (to transparent) that isn't the "content-aware fill"
    5. Save the resultant image file. I call it the "matte".
  3. Import the image file and place the matte in the timeline above the original video. Just like magic, the object is gone (hidden by the mostly-transparent matte).
  4. But wait, there more... It turns out that the brightness of the surrounding area in the picture is probably not constant. So, as the video progresses, if the surrounding area in the video gets brighter or darker (because the camera is doing some sort of auto-brightness thing) then the matte appears to be a sharply-defined darker or brighter area against the image. So (and this is the time-consuming part):
    1. Select the matte layer
    2. Apply the Lumetri color effect
    3. Go to the beginning of the timeline, and enable keyframes on color wheels/mid-tone brightness.
    4. Switch to the Color workspace, and display the color wheels section
    5. Play (or manually scrub) the video. As it progresses, you will see the luma in the scope move up or down as the camera adjusts its brightness. At the same time, you will see your matte become visible against the background. Slide the mid-tone slider up or down to bring everything back into equilibrium.
    6. Repeat that process. Again. And again. And play it back and fix the places where you didn't get it quite right...

 

So... This works, but the brightness thing takes a lot of work to sort out. I asked a separate question and, apparently, there is no way to automate things so that the area around the non-transparent matte remains at a constant brightness.

 

Another option, which wouldn't be as seamless but might be "good enough" and a lot less work, would be to define a marquee around the objectionable object in the image and make all of the colors in that object match the surrounding color. But I haven't found a way to do that well enough that the area doesn't stand out worse than the original "objectional object" in the image.

 

Any suggestions?

Adobe Community Professional
Correct answer by John V Knowles - WM | Adobe Community Professional

There is a Content-Aware Fill option in After Effects now; if you're already familiar with the one in Photoshop then give the one in AE a try. You draw a maks around the object and it tries to fill in the blank. It may also try to replicate the lighting shifts in the footage, though there's no guarantee that it will work 100% perfectly. 


WM

TOPICS
Effects and Titles, How to

Views

65

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Sep 21, 2020 0
Adobe Community Professional ,
Sep 21, 2020

Copy link to clipboard

Copied

There is a Content-Aware Fill option in After Effects now; if you're already familiar with the one in Photoshop then give the one in AE a try. You draw a maks around the object and it tries to fill in the blank. It may also try to replicate the lighting shifts in the footage, though there's no guarantee that it will work 100% perfectly. 


WM

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 21, 2020 0
Explorer ,
Sep 22, 2020

Copy link to clipboard

Copied

Oh... After Effects... I've never even so much as installed it. I did some Googling to see how to use it in conjunction with Premiere Pro, and I found a half dozen different ways to share work between PP and AE. So, for this use case, what's the easiest way for me to start in Premiere Pro, take my clip into After Effects, and bring the edited clip back into Premiere Pro? Do I need to create a separate After Effects project, or can I do something similar to the way I can (more or less) seamlessly take audio into Audition and get the edited results back in my Premiere Pro project?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 22, 2020 0
Explorer ,
Sep 22, 2020

Copy link to clipboard

Copied

A little more Googling... I found this helpful video that explains Premiere + After Effects:

https://youtu.be/PLscuuuY0zk

 

Check out my Premiere Pro Course: https://www.udemy.com/course/premiere-pro-course/?referralCode=AF659E18BEF06A7F4955 Subscribe to see more Adobe related con...

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 22, 2020 0
Explorer ,
Sep 23, 2020

Copy link to clipboard

Copied

The good news is that the process of taking a clip from Premiere into After Effects and applying content-aware fill on an object in the clip is quick and easy. And the results are impressive. The object simply disappears, and the "fill" is calculated frame-by-frame, so it doesn't matter if the the lighting is changing or even if the object is something like a person walking on the beach with a dynamic background. And with a little keyframe magic, the mask that defines the area to be filled can be animated to track a moving object in the clip.

 

The bad news is that it's totally unusable because After Effects takes many, many hours to compute the effect over a 4-minute, low-resolution clip. My clip started out at 1920 x 1080. I reduced the resolution to half of that (since the clip is being scaled in the final production anyways, so I don't need all that resolution). But even at 960 x 540, with a tiny area of the clip selected to be filled, I left After Effects running overnight and, after 12 hours it had completed the "analyzing" step (that took maybe 5 hours) and it was halfway through rendering frames at about 5 seconds per frame. This on my fairly powerful desktop PC with a fancy GPU and a powerful CPU. Oh, and it had created about 20 GB of files, apparently one per frame. Presumably, if I let it finish, it would produce another 20 GB of "frame" files.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Sep 23, 2020 0