Duplicate detection problem with photo downloader

[johngh]

Hi - I'm in the trial period for Elements 2020 and think there's a problem with the photo downloader duplicate detection.

All works well if you accept all the defaults and import the same photo twice from an SD card - the second time it is detected as a duplicate. In this case the modification time and file size of the imported file is the same as the original.

However if you check the option to preserve current filename in XMP (which I want to do) duplicate detection fails. I suspect this is because the filesize and modification date of the copy is different (metadata has been written to the file) and one or other of those are being used to decide whether it is a duplicate.

This would be a problem for me, and I think is probably a bug that would be triggered by some of the other options e.g. applying metadata or fixing red eye on import. Surely the catalog should remember the original file attributes, rather than relying on reading the attributes of the copy, and compare attributes of files on the SD with the remembered values. 

Is this a known problem (surely it must be) and are there work arounds or plans to fix?

[MichelBParis]

"Is this a known problem (surely it must be) and are there work arounds or plans to fix?"

I have been following the issues with duplicate detection in this forum and in the feedback forum for more than 15 years.

I can say that there has always been many issues, but I have never seen the filename in xmp mentioned as a possible problem. The two main criteria you have quoted are indeed file size and date_taken, but there are other ones which have never been disclosed in detail. Storing filename does not change file size significantly, I can't imagine it would be the source of your issue.

The number of complaints about duplicate detection has become quiter rare. You would need to describe the import workflow in details to make it reproducible and to give the most information about your own setup, particularly your OS precise version. I don't think Adobe will look into the problem otherwise.

However, it's a good thing to discuss such issues in this forum to better understand how detection works and if it's possible to avoid duplication and to find workarounds.

[johngh]

Michel, many thanks for your reply. Further details as requested:

Systems info from PE help:

Elements Organizer 18.0.0.0
Core Version: 18.0 (20190827.m.136736)
Language Version: 18.0 (20190827.m.136736)

Current Catalog:
Catalog Name: test1
Catalog Location: C:\ProgramData\Adobe\Elements Organizer\Catalogs\test1\
Catalog Size: 155KB
Catalog Cache Size: 500.2KB

System:
Operating System Name: Windows 10
Operating System Version: 10.0
System Architecture: AMD CPU Family:15 Model:1 Stepping:0 with MMX, SSE Integer, SSE FP, SSE2
Built-in Memory: 32GB
Free Memory: 23.6GB

Steps to reproduce issue:

1. Initial setup

An SD card in a card reader D: contains a single file RIMG0028.JPG that has not yet been imported by PE.

File attributes (from Explorer - Properties) include

Modified: 10 Jan 08 20:16:34
Size: 1,686,241 bytes

2. Confirm that duplication detection succeeds when importing with default options

- Import from Camera or Card Reader

- Select source device D:\<Camera or Card Reader>

The image is shown in the downloader as selected

Options are all default (except that location is set to a folder on the internal SSD C:\Users\john\photo test\)

- Click "Get Media"

The downloader copies the file to C:\Users\johng\photos test\2008 01 10\RIMG0028.JPG
with exactly the same size and modified date.

- Repeat the import process above

The file is not selected for import confirming it was detected as a duplicate.

3. Return to baseline

- Delete image from catalog and hard disk

4. Confirm that duplication detection at import fails when filename is written to XML

- Import from Camera or Card Reader

- Select source device D:\<Camera or Card Reader>

The image is shown in the downloader as selected

- Check on "Preserve Current Filename in XMP"

- Click "Get Media"

The downloader copies the file to C:\Users\johng\photos test\2008 01 10\RIMG0028.JPG
and the metadata info in PE confirms that a PreservedFileName attribute has been added.

Now the modification date and file size of the copy differ from the original.

Modified: 25 Jan 2020, 18:37:20
Size: 1,667,004 bytes

(In this case it shrunk slightly, I've also seen it increase slightly).

- Repeat the import process as above

The image appears in the downloader as selected and "Get Media" imports it as a duplicate with filename RIMG0028-1.JPG

So duplicate detection has failed in this case.

As said, I suspect this is because the downloader looking for an exact match between the candidate file and existing files in the catalog on either filesize or modified date. As it altered the file on import it fails to find one.