4
Replies
Exiftool has been recommended to analyse the files and I am going to look into that.
In the meantime, I would be glad of any advice about:
a) how to reconcile the new catalog against the old messy file structure
b) understanding why importing files identifies some as already present and skips them, but imports other which also seem to be duplicates.
Thanks in advance,
Billy.
By @Billy5FE5
Hi Billy,
First, abut exiftool. It's indeed the industry reference for metadata management. Most users don't use it directly, since it's a command line editor, but you'll find many softwares integrating its features in a handy graphical interface (such as exiftoogui ).
As suggested in the post by John R Ellis, the problem is mainly that there are much too many interpretations and implementation of data management with different softwares. I am convinced that the solution to your problem lies in a difference in how the Explorer and the Organizer manage the real date_taken in various circumstances. That's a topic which has been discussed widely since 20 years. John R Ellis has left PSE after PSE8 and is now the expert for those matters in Lightroom classic. I am sure that a closer check of the non matching duplicates in exiftool or similar will reveal the difference when comparing the data in the Organizer catalog. The problem will be to explain how the different workflows have created that change. In all cases, there has been metadata changes when saving or copying files between:
- different softwares (including the Explorer)
- different backups
- different catalogs
Somewhere in the process, there is some specific metadata which is missing and where the organizer tries to find the equivalent metadata field. A number of real bugs have been reported, and different interpretations of the exif standards are still too frequent.
Now, if we come back to your particular search, we may hope to find the real reason of the failure to find 100% of the duplicates. But it will be linked to the particular history of the files on your computer(s); only you can know.
Since I can't give you the precise explanation, I want to stress my own experience since the start of the Organizer 20 years ago.
- I have had duplicates issues like everybody else, but duplicates in a catalog without explanation are very rare and not significant.
- I don't use watched folders because I prefer to try re-importing some folders manually and I have never had significant issues with the duplicates filtering.
So, my earlier suggestion did work for me, especially for clearing duplicates on the computer but not in the catalog. In some instances I was able to find a big number of duplicates created from the import or the copy/restore of batches between different computers or catalogs.
From experience I can say that working on a single catalog, sharing my catalog+library on an external drive for different computers and the like do really protect against problematic copies.
-
Thanks @MichelBParis .
I'll go away and look at this systematically and in depth.
My first step was to copy the two folders in question in to a seperate test area then import them in to a new catalogue. When I did that, PSE identified all 97 as duplicates which seems in line with your belief that the root is differences in how file explorer and organise manage dates.
It's going to take me a while to get to know the tools but I am going to look through the differences between the files and I expect that I'll see a pattern (eventually).
Thanks again for your advice.
By @Billy5FE5
We are focusing on the ability of the downloader to catch duplicates, but there has been real bugs/issues with geotagging ( rounding errors...) and time zones/DST. More recently when downloading present duplicates in smartphones or certain cameras. Something I have not seriously tested: what if you want to import a file which is a strict copy of another but with different tags? Same date_taken, but possibly nearly the same size? For me, they are not duplicates and you would risk skipping the good one with the tagging.
Another interesting fact is that while the organizer is built to avoid duplicates, it offers the command to create them voluntarily. Menu File > Duplicate. The duplicate is created in the same folder with a ' copy' suffix. You can move it elsewhere if you want. Both the original and the duplicate are imported in a new catalog !
AdChoices