Why does Import skip some duplicates but imports others when they all look like duplicates to me

Exiftool has been recommended to analyse the files and I am going to look into that.

 

In the meantime, I would be glad of any advice about:

a) how to reconcile the new catalog against the old messy file structure

b) understanding why importing files identifies some as already present and skips them, but imports other which also seem to be duplicates.

Thanks in advance, 

Billy. 

By @Billy5FE5

Hi Billy,

First, abut exiftool. It's indeed the industry reference for metadata management. Most users don't use it directly, since it's a command line editor, but you'll find many softwares integrating its features in a handy graphical interface (such as exiftoogui ).

As suggested in the post by John R Ellis, the problem is mainly that there are much too many interpretations and implementation of data management with different softwares. I am convinced that the solution to your problem lies in a difference in how the Explorer and the Organizer manage the real date_taken in various circumstances. That's a topic which has been discussed widely since 20 years. John R Ellis has left PSE after PSE8 and is now the expert for those matters in Lightroom classic. I am sure that a closer check of the non matching duplicates in exiftool or similar will reveal the difference when comparing the data in the Organizer catalog. The problem will be to explain how the different workflows have created that change. In all cases, there has been metadata changes when saving or copying files between:

- different softwares (including the Explorer)

- different backups

- different catalogs

Somewhere in the process, there is some specific metadata which is missing and where the organizer tries to find the equivalent metadata field. A number of real bugs have been reported, and different interpretations of the exif standards are still too frequent.

Now, if we come back to your particular search, we may hope to find the real reason of the failure to find 100% of the duplicates. But it will be linked to the particular history of the files on your computer(s);  only you can know.

Since I can't give you the precise explanation, I want to stress my own experience since the start of the Organizer 20 years ago.

- I have had duplicates issues like everybody else, but duplicates in a catalog without explanation are very rare and not significant.

- I don't use watched folders because I prefer to try re-importing some folders manually and I have never had significant issues with the duplicates filtering.

So, my earlier suggestion did work for me, especially for clearing duplicates on the computer but not in the catalog. In some instances I was able to find a big number of duplicates created from the import or the copy/restore of batches between different computers or catalogs.

From experience I can say that working on a single catalog, sharing my catalog+library on an external drive for different computers and the like do really protect against problematic copies.

-