Skip to main content
P
Philip22255932eaa9
Participating Frequently
February 16, 2024
Question

Editing Video - wanting to remove lens smudges and dirt from video

  • February 16, 2024
  • 2 replies
  • 5410 views

I have travel video taken with a 360 camera on the front of a motorcycle, where bugs or a general foreign object landed on and stuck to the front lens.  I would like to edit the video (frame by frame if-as necessary) to correct this situation.

What I have tried - I recently purchased a subscription to Photoshop 2024, using it to edit video.  I do understand that Premier is the primary Adobe application for video, but having used PS Elements for a long time, I thought to try it first before considering other options.  I have used the 'remove tool' with some success, removing an insect smudge located on the video in clear blue sky.

The MAIN problem I need help with is correcting the circumstance where the background is changing and the smudge or insect (the anomaly) is stationary on the lens but moving across the video, where the background is in motion.  IMPORTANT - where the anomaly moves, frames of video BEFORE and AFTER a frame are clear of the anomaly in a specific location.  My idea is to COPY a section of a previous frame where the anomaly does NOT cover a location and PASTE it into that location in following frames.

I was able to use the 'lasso' tool and copy function to select the area clear of the anomaly in 'frame 1', but when I attempt to paste it into a 'frame 2' using every paste option (paste, paste special - into, outside, etc.) I get nothing.  I can see that a new layer is created, but no method I have found thus far pastes the copied section into the new frame.  I have tried rearranging the layers in the sequence, flattening, etc. but the copied area never appears in the new frame.  Often, the entire video is reduced to one image (?).

I understand that PS 2024 may not be the best application to edit video, so I am reaching out to video editors to find a basic way to accomplish what is, admittedly, a bit of a complicated process.  Would the Premier program accomlish this method more easily?  The smudges and anomalies are keeping me from completing a project, and I have trying to find a method to satisfactorily correct this video for months now.

Sorry for the novel, but I also wanted to emphasize that my video editing is primarily limited to just corrections and basic functions - I typically do not stylize images or video, so I would also like to know if PS Elements (which I am very accustomed to using) or Premier Elements 2024 as more basic programs have the AI 'remove' tool and could accomplish the 'copy and paste' process that I am trying to accomplish.  My Mac Mini is an older model, and while a new M2 purchase is planned, I am thinking that PS Elements - Premier Elements might be less taxing on the older mini.

If someone out there has a reasonably straightforward method of accomplishing what I need to do here, please respond, and thanks!

    This topic has been closed for replies.

    2 replies

    Conrad_C
    Community Expert
    Conrad_CCommunity Expert
    Community Expert
    February 19, 2024

    I spent a little more time looking at it, and there seems to be a third approach (other than the one you were trying, copy and paste, and the one I was thinking of). Have you tried the approach shown in the demo below?

     

    I realized that there is a flaw in the copy/paste method in that the selection includes a fixed set of background pixels , so in subsequent frames there’s a chance they won’t match up with the changing background and the edges will be obvious. I played around a little and it seems to work fine if you set the Remove tool to Sample All Layers and paint it on a higher video (not still) layer, so all you have to do is go frame by frame and paint the correction in each one. That lets the Remove tool adapt the healing to the specific pixels in each frame.

     

    The example below removes a reflection from a train window in front of a constantly changing background, painting the Remove tool correction on new video layer Layer 1. Before choosing Layer > Video Layers > New Video Layer, it’s important to click to select the Video Group but not any clips in it.

     

    This works because we know Photoshop can’t paint on the actual video layer (for the reasons I mentioned earlier), but if a new empty video layer (not still layer) is created, Photoshop can merge the corrections on that new video layer with the original video, when you have Photoshop render a new video from this sequence.

     

    P
    Philip22255932eaa9Author
    Participating Frequently
    February 19, 2024

    Okay, I am trying this now - thanks!

    I did try the remove tool frame by frame on the original video timeline, and the tool would work, but given the limitations of a tool designed to be used primarily for still images, each 'remove' attempt would render a different-looking corrected area, so when running the video, the corrections look as obvious as the smudge.  With a small spot moving horizontally across the video,  each use of the remove tool creates a new corrected area that does not need to match the preceding frame background.  The problem I am having is that I am moving forward in the timeline, either walking a trail or riding a motorcycle, where the camera is mounted on the bike or held stationary, so for two or three frames at a time the smudge overlaps or partially overlaps the same location.  For this to work, I have to get corrections to match the existing background as the smudge moves past the area.


    The idea would be that I'd "transform" the copied area (skew, perspective, distort, etc.) to "match" the copied area to the new frame, if I could get the method to work as it does on a still image.   The smudges are not 100% opaque, so I can see detail "below" the smudge well enough to 'line up' the pasted layer (I think!) and I would use some feathering in the copy-paste process so that the edge of the pasted area would hide (ideally).  Can this be done on a layer?  I guess I'll try that too.

    Will adding the layer somehow change how the remove tool performs its algorithm?  I guess I'll see.  I have set it up as per your instructions and the video.  Good stuff.

    Ultimately, I think I'm going to need to get an M2 and learn After Effects!  Thanks again, Conrad.  I'll let you know how this turns out. 

    P
    Philip22255932eaa9Author
    Participating Frequently
    February 19, 2024

    Also, a beginner question - is there a method to move the video frame by frame in Photoshop by using keyboard keys?  Other applications use the right-left arrow keys to do this, but that does not work in Photoshop.  Having to move the cursor from where I am making a correction on the frame down to the single frame arrow and back is adding a lot of time to this frame by frame correction thing.

    Conrad_C
    Community Expert
    Conrad_CCommunity Expert
    Community Expert
    February 16, 2024
    quote

    The MAIN problem I need help with is correcting the circumstance where the background is changing and the smudge or insect (the anomaly) is stationary on the lens but moving across the video, where the background is in motion.  IMPORTANT - where the anomaly moves, frames of video BEFORE and AFTER a frame are clear of the anomaly in a specific location. 

    By @Philip22255932eaa9

     

    Yes, this is a common problem, but when the content moves around the frame over time, the correction is not easy to animate across a large number of successive frames in Photoshop. But as you suspected there are other tools (more on that later)…

    quote

    My idea is to COPY a section of a previous frame where the anomaly does NOT cover a location and PASTE it into that location in following frames.…but when I attempt to paste it into a 'frame 2' using every paste option (paste, paste special - into, outside, etc.) I get nothing.  I can see that a new layer is created…

    By @Philip22255932eaa9

     

    It sounds like that’s working as expected…copy/paste is not normally how this would be done, partly because Photoshop doesn’t alter the content within video frames, and partly because that’s a lot of manual frame-by-frame pasting. If I was to do this in Photoshop, the correction would be on a separate layer, and then its Transform property animated over time (using keyframes). But having to manually create a different keyframe every time the content moves is still extremely tedious and time-consuming, and so it’s not the recommended method.

     

    Premiere Pro does have a more automated way to do that, using its Motion Tracking feature. The corrected smudge could be attached to a motion tracking element that automatically follows the smudge around the screen.

     

    But probably the easiest/fastest way to fix this (because it’s the most automatic) would be using the Content-Aware Fill feature in the specialized visual effects application Adobe After Effects. This is like the Remove tool combined with Motion Tracking, so you can cut out an unwanted object in After Effects, click a button, and it will follow that around the screen, removing it from every frame.

     

    Adobe help article:

    Remove objects from your videos with the Content-Aware Fill panel (in After Effects)

     

    Quick video demo:

    P
    Philip22255932eaa9Author
    Participating Frequently
    February 16, 2024

    Conrad - thank you for the knowledgeable response!  I had mentioned the older Mac Mini that I have, because I actually have tried the After Effects program, but my computer was really not up to it.  It took a LONG time to process the corrections I tried, and I was continually aware that the computer was on the edge of crashing each time I tried something using the AE program.  PLUS, while the fact that After Effects is a very capable program, it is mind-numbing to get up on when all you're trying to do is clean up a smudge or a bug on the lens.

    Even using the 'remove' tool in Photoshop required a frame-by-frame editing process, and I understand very well how tedious that process is, but I would be willing to do it if I could get a QUALITY correction using a copy and paste method.

    Corrections of this type take an accuracy that I think only an AI process may be able to accomplish.  Even using the very effective 'remove' tool in PS leaves artifacts of its algorithm that change from one frame to the next, so it is obvious when running the video that edits were tried - it's almost as obvious as the smudge itself.  Copy and Paste in the traditional Photoshop method could accomplish this better I think, with a little 'transforming' the copied area to match the background area in the pasted frame, and I would grit my teeth and do this frame by frame if after hours of working I could get satisfactory results.

    I used an Insta360 camera on these videos (a week's worth), and if you know the camera and the application the company provides, you can direct viewer attention in any direction in a 360 degree view.  My idea is to work around the 'face forward' view where possible, but ultimately, you have to give the viewer a face forward view at some point, because it's really necessary for a what?... a satisfying viewing experience I guess you'd say, so I would do the 'frame-by-frame' in long enough segments of a face-forward view to accomplish that.  It is only in the 'face forward' view that bugs and smudges are apparent, for obvious reasons.

    Anyway, thanks again Conrad - I will look at the video and study your suggestions and try to assimilate.  In the meantime, I still would like to know if PS Elements and/or Premier Elements would accomplish what amounts to basic functions (copy-paste, AI remove, etc.) on video, even if this would require editing frame by frame.  Since I know the PSE program, and I am thinking Premier Elements may be similarly a lower level of complication than the "pro" version, this may make a complicated process less complicated.  If anyone out there who knows can comment on this, please do.  Thanks, Phil.

    Conrad_C
    Community Expert
    Conrad_CCommunity Expert
    Community Expert
    February 17, 2024
    quote

    In the meantime, I still would like to know if PS Elements and/or Premier Elements would accomplish what amounts to basic functions (copy-paste, AI remove, etc.) on video, even if this would require editing frame by frame.

    By @Philip22255932eaa9

     

    I don’t know very much about those applications, so I hope someone else is able to fill in that part of the answer.

     

    Having an Intel-based Mac mini can be a limitation; the Apple Silicon Macs are certainly a big step up in performance.

    Terms & ConditionsAccessibility statement
    Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices