LR CC Classic does not see duplicates -

[dhmiller]

I know this has been covered before, but I use a program called WhatSize to identify duplicate files and on one hard drive, I have over 10,000 duplicates. These files are not in the same folder (MAC O/S) but have the exact name, size and dates as the originals yet LR CC C does not recognize them. This is not a new problem with this release.

Is there any way to force LR to better handle duplicates?

[JP Hess]

How are you expecting Lightroom to handle duplicates? There is an option in the import dialog instructing Lightroom to NOT import suspected duplicates. Are you choosing that option? What do you want to have happen?

[dhmiller]

I am expecting LR to recognize the duplicate files upon Import and not add them along with the files that are not duplicated. I am trying to make a master catalogue of all the files on one hard drive (for, ultimately, copying to a second drive) and I do not want two copies of any of them in the catalogue. I had assumed this was the expected result of choosing Do Not Import Suspected Duplicates.

[JP Hess]

I'm afraid you have lost me. If the files have the exact same name, are the exact same size, have the exact same date as the original, and you have instructed Lightroom NOT to import suspected duplicates then Lightroom isn't going to import those duplicate images. That is the expected behavior.

[mhshoot]

You should not be trying to help as you clearly aren't qualified. Lightroom has had a problem detcecting duplictaes for a long time.

[JohanElzenga]

You should not be trying to help as you clearly aren't qualified. Lightroom has had a problem detcecting duplictaes for a long time.

By @mhshoot

If you really need to make unfriendly remarks about somebody who is just trying to help, then at least check that you do not reply to a 4+ years old discussion...

[cmgap]

As Jim has written Lightroom will not import duplicates if you have 'Don't Import Suspected Duplicates' checked.

Have you done a search on an image name and made sure that the images are exact duplicates? 10,000 duplicates is an uncommon scenario.

[dhmiller]

Apparently this was not clear:

"I do not want two copies of any of them in the catalogue."

I do not want the duplicates to be imported but they are being imported despite the fact that I have Don't Import Suspected Duplicates enabled.

Maybe I'll upload an example of two identical files, one of which is in the catalog already and the other LR is trying to import even though the command is enabled.

I should have been clearer on the following: not all of the 10k duplicates that WhatSize is reporting are images (though most are).  There are over 200k files on this computer...all my photos dating back to 2009.

[cmgap]

Are the file extension types exactly the same? JPGs or other? If you have RAW files and jpgs that will make a difference even though they are the same image.

Please upload a screen shot of the files and metadata. Or share the actual files in a dropbox folder and send the link by clicking on my screen name and messaging it.

[dhmiller]

The files are 100% identical except that they are not in the same folder. I found an LR plug-in that identifies duplicates (that LR missed upon Import) so I went ahead and imported everything then used the plug to delete all the duplicate files LR imported. Someone at Adobe should look at that code..

Thanks for the help.

[cmgap]

There is a way that Lightroom will miss duplicates. If the box as shown in File Handling is unchecked the first time or second time they were imported. The only way I can import duplicates is to uncheck the box and force duplicates into the catalog. Perhaps other ACPs can come up with other scenarios.

[dhmiller]

Well I'm certainly not <looking> for ways to import duplicates - quite the opposite. But I am 100% certain that LR missed a number of dups and I had to use the plug-in to remove them. Don't know why there would be a plug-in to remove dups if LR was 100% perfect at doing that, though I imagine someone might have unchecked Don't Import Duplicates then changed their mind.

Anyway, my issue is resolved.

[davef56729325]

What plug-in did you use?

[davekfly]

yeah what plugin please! !

[davef56729325]

This is the one I used and it's free..I would give a donation though

http://www.bungenstock.de/teekesselchen/

I tried the Lightroom plug-ins.com but it didn't work

[Jao vdL]

The don't import suspected duplicates feature is extremely conservative in identifying duplicates. I've seen it not work correctly if I have imported the exact same memory card through different card readers for example or when I had already imported the exact same files into CC on another machine and synced into the Classic catalog. It will often not recognize duplicates. My guess is that Adobe errs on the side of caution to not have people lose images.

[Pomax]

Late to this question, but my solution in these cases (and they happen a LOT) is to use a dedicated duplicates finder (in my case, Duplicate Photo Finder). I periodically run it after imports, make it delete the duplicates it finds (because there is no reason to keep them on disk: just have the first instance in the appropriate LR collections, and tagged with the right keywords), and after that's finished and I'm pretty confident I no longer have duplicates anywhere, I synchronize my folders in LR, with _only_ the "remove missing photos from catalogue"  option checked.

(technically, I don't, I run a catalogue scrubbing script that does the same thing without having to run LR, because the catalogue is just a sqlite3 file, and removing non-existent file entries is an almost trivial set of "remove entry from each table where image_id is in the list of deleted images", doing the same thing as that synchronize folder option. However, where LR takes forever to sync, directly updating the db file using https://github.com/pomax/lightroom-catalog-helper takes <100ms)