How to eliminate duplicate copies after doing a sort

[Spaceman2011]

I did a filter/sort across 100's of folders then exported the results to a separate folder.

I later noticed way too many duplicates in the new folder.

Does Lightroom have a feature to automatically remove or eliminate duplicates when doing such a process??

If not, any other ideas??

[web-weaver]

LR has no feature to recognize duplicates if they have a different file name. LR cannot recognize dupicates according to image content.

I found an old forum entry where something called dupeguru was mentioned that supposedly would recognize duplicates based on image content. The post said that it would not work in LR so you had to apply it on its own.

Check it out: http://www.hardcoded.net/dupeguru_pe/

I have no experience with this program; I don't know if it works. Don't even know if the link is OK.

WW

[Spaceman2011]

Thanks for the quick response.

My problem is with the same name, not different names. The LR sort group has lot of duplications with the same name.

Seems like they could easily reject duplications during that process!!

That's what I was hopping for.

Thanks Again.

PS: How do you edit your original question??

[Spaceman2011]

Hi WW

Thanks for your quick response!

Again, I am currently trying to learn Lr on a 30 day free trial. So far,

I am really liking this software.

However, I have just discovered that Lr does not save the

changed-file-data to Explorer!! Apparently all of the data that contains

changes stays only within the Lr software package. That means that in

the future I will not be able to view my latest work in Explorer, Photo

Gallery, any other photo software that I have, or view the changed work

on my wife's pc via the network. I don't recall ever having such a

problem before.

Can you please verify that this is true, or tell me what I am doing wrong.

If it's true, hopefully there is an easy & simple work around.

If not what is the best way to live with it??

Thanks

Spaceman2011

[bobmeyerweb]

Spaceman, you're basically correct that LR doesn't write your changes to the original file itself.  LR is a non-destructive editor, which is really a good thing. It means that if you screw up an edit, you can always go back and start over. Or if you decided to convert the image to B&W, and now want to go back and re-edit it in color, you can do that. There are, however, a couple of ways to do what you want.

The easiest way is to use the Export dialog. This will create a copy of your image(s) with all the edits applied. Those copies can then be viewed in any image viewer program. You can export as jpeg or tiff files. Based on your question, I'm guessing you typically shoot and process jpeg files (since most image viewers won't display raw files), so LR will create new jpegs, with all your edits. You can give these files new names on export, or write them to a different folder. You can even automatically import the images into the LR catalog if you want. (I don't, as if I ever want to re-edit the images I'll start with the original file.)

LR can also write the changes to what's called a "sidecar" file, that you can copy along with the original image file.  But I don't think very many apps know what to do with that file, either.

[CSS_Simon]

I guess it depends what you want to do with the duplicates.  I mean: presumably, having found duplicates, you're going to have to go through them individually to choose which to keep? 

If all your photo folders are under one common top-level folder (if not, you can always move them - but do it inside LR by dragging folders) then you can view all files in the catalog in file-name order.  In Library, make sure Library menu, "Show Photols in Subfolders" is checked, then simply select the top folder and make sure the sort is by name.  Go through and you'll see duplicates with the same name before you even see the name (I mean, they'll look he same).  Tedious, but if you've got to go through choosing which to keep it's going to be tedious anyway. 

[Spaceman2011]

Hi CSS Simon

Many times in the past I have had a need to sort across 100's of folders

to search for a particular tag or ranking. While copying the results to

a new folder, I found that all duplicates would automatically be

eliminated. So far, I can't find a way to do that with Lr. Bummer!!

Thanks for your response.

Spaceman2011

[Hal P Anderson]

Spaceman,

In Lightroom, you can go to the All Photographs collection and sort by file name. That will at least bring all the duplicates next to each other. Having duplicate images in your catalogue can be a bother. How did they get there? There's a box you can check to prevent duplicates from being imported.

And yes, Lightroom doesn't change your images. That's the point of being non-destructive. If you want images to share, you create them on demand through the export process (or Print, Web, or Slideshow). The exported images can be treated as temporary in the sense that you don't need to import them back into Lightroom because they can always be re-created.

Lightroom is kind of like a marriage (Not my metaphor.). When you use it, it works best with some commitment on your part to use it as your sole entry to your images. It is a good partner in most respects .And like all good spouses, it has its flaws.

Hal

[Spaceman2011]

What great responses!!

Thanks for such excellent replies!!

[lgraffxxx]

Once all the duplicates are all sitting next to each other, is there a batch process to determine which file is the real thing and eliminate dupes and bad links?

[dj_paige]

Once all the duplicates are all sitting next to each other, is there a batch process to determine which file is the real thing and eliminate dupes and bad links?

I do not think Lightroom has any such capabilty. Nor do I think any software can determine which is the "real thing" and which is a "duplicate", I would imagine that has to be a human decision.

[lgraffxxx]

Thanks. Will just have to go through them individually. Dupes came in while "importing from other catalogs" to build a master catalog. Are there settings (other than "do not import duplicates") that should be used to prevent this. (Hopefully, I won't need to do this until next year, when Lightroom 6 comes out; think a better process could be developed.)

[dj_paige]

(Hopefully, I won't need to do this until next year, when Lightroom 6 comes out; think a better process could be developed.)

While I am glad you are hopeful, I don't know what you base this hope on. Neither you nor I know what will and will not be in LR6, but I haven't noticed a large number of people calling for better handling of duplicates. If I had to bet, LR6 will not have better handling of duplicates (nor will LR7 or LR8 etc)

[renderdc0]

I believe you can query duplicate files by searching the folder that contains the images by typing in the search field *duplicate file-name.

In my case I downloaded duplicate memory cards into my lightroom library and realized Lightroom had automatically resolved the duplicates by adding a -2 behind the original file-name...ex: rob_3455.NEF  --> lightroom changed to rob_3455-2.NEF On a Windows machine. You can query part of a file name within a certain folder by typing an asterix in front of the variable in a search field.

Hope that helps.

[Painting Light]

Use a program called Gemini from the app store. Finds duplicates in any part of your mac. One of the best features of this program is that it finds duplicate photos which have been renamed. Absolute life and space saver. Gemini: Best Mac Duplicate File Finder. How to Remove Dulicates on Mac OS X

[larsonv83671003]

try this DuplicateFilesDeleter

[steve185]

Under the image area, next to the A/Z option, Sort "Added Order" as presumably you didn't import them all at the same time. This will sort your images into 2 groups. Shift select the first and last images of one group and delete them.

[ArthurV]

My workflow for this:

in the Library on top select the text option and search for any images that filename contain -2 (this is the wai LR rename duplicate) then it will display all image with the -2 (duplicate)

[Jose Lamarque]

Thank you!