• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
9

2023 Elements Organizer import causing duplicate photos

Explorer ,
Jan 17, 2024 Jan 17, 2024

Copy link to clipboard

Copied

Hi folks, long term user, love the product.  My wife is unhappy (so me too!) with all the photos being duplicated after PSE23 organizer import from iphones (via USB).  I have been reading thru all of the related posts and cant seem to find a match to my scenario/artifacts, many posts may document workarounds (sd card, or use another app) or show no resolution.  Please help me parse this issue here, ty in advance for replies/recc's.  My baseline configs are PSE23 and PSE22, two windows computers (His and hers! - one win10, one win11), 1 networked synds418 Nas for storage.  I use "delete origonals", The result is duplicate photo's in my NAS storage folders.  Organizer seems to run as expected, but hangs after import, (i dont see the delete photos step anymore), I must force close PSE23.  After reopening/reboot?) PSE23 organizer I see a quick flash of "importing photos" (multiple photos). I still never see the delete photos step (like I used to)...    I'll pause here and add more embedded phootos on a test process next...   TY!!!!!   

DataGeek
TOPICS
Crash or freeze , How to , Import and export , Organizer , Problem or error , Windows

Views

273

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Jan 17, 2024 Jan 17, 2024

Copy link to clipboard

Copied

Elements Organizer 21.0.0.0
Core Version: 21.0 (20230329.Git.master.77124fa)
Language Version: 21.0 (20230329.Git.master.77124fa)

Current Catalog:
Catalog Name: My Catalog 1
Catalog Location: C:\ProgramData\Adobe\Elements Organizer\Catalogs\My Catalog 1\
Catalog Size: 82.4MB
Catalog Cache Size: 1.3GB

System:
Operating System Name: Windows 10
Operating System Version: 10.0
System Architecture: Intel CPU Family:6 Model:7 Stepping:1 with MMX, SSE Integer, SSE FP, SSE2
Built-in Memory: 31.6GB
Free Memory: 17.2GB

DataGeek

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Jan 17, 2024 Jan 17, 2024

Copy link to clipboard

Copied

I ran test import, no duplicates detected on impiort screen (not highlighted), but did show both heic and mov files (my iphone has "keep origonals" selcted to "automatically transfer photo's and videos in compatible format".  iphone showed 92 recents, Organizer showed 198 image and videos.  Started organizer, seem to run correctly"

PhotoGeek2121_0-1705527029262.png

then got a blank dialogue box for approx 10 seconds, then it closed as well (maybe delete step), the sort by "newest" view never updated to this most current unload.  I closed PSE, then reopened, as soon as I reopened organizer, I got the "secondary" import window (as follows):

  

PhotoGeek2121_1-1705527352240.png

when this completed (to 198), the "Newest" view did update.  I then checked my NAS and found no duplicates (I did find  the expected mov and heic versions of the same picture.).   

DataGeek

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Jan 17, 2024 Jan 17, 2024

Copy link to clipboard

Copied

You cannot use the same catalog with different versions of Elements.  You haven't told us enough about your combined workflow for me to fully understand why or where the duplicates are occuring but I suspect that is the root cause.  (Adobe also does not recommend using a NAS for your catalog(s)).

 

Added:  And just to make myself clearer, the catalogs that you and your wife are using in the different versions are different, although they may be using the same media.  If you are each importing images into the respective catalogs from the same device or card, there will undoubtedly be duplicate files created.  If I have misunderstood your combined workflow, we need more details of what you and your wife are doing with your respective catalogs.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Jan 17, 2024 Jan 17, 2024

Copy link to clipboard

Copied

ty Greg, initially our import strategy was to unload wifes iphone on her computer, with her local PSE22 (and local catalog) to her c drive, saved to shared NAS folders.  My iphone/ipad, my pc, my local PSE23 and catalog, to shared nas destination.  This model seemed to work until a few years? ago, when the duplicates and hangs started.  Now in the last few months, we cant unload her iphone on her pc (pse22) anymore.  Organizer says no valid files found.  itunes/explorer can see the phone but organizer not happy, may be an artifact, We have now started to unload both iphones from my PC, my local PSE23 and organizer catalogue.  we are not storing the catalog on our nas.  Just the imported images and videos are being stored on shared nas folders.       

 

The photos on each device are unique to that device, organizer just channels it to the shared nas server, each iphone can use the same organizer to get it to the nas, recent catalog views will be specific to that iphones import.  The duplicates I an having trouble with are on my server. 

 

ps..    in the test run I just ran, NO duplicates were produced.  I dont like the fact that a final "import" dialogue boxes were initiated by organizer after an app closure and restart, but it may be an artifact..   I will continue here to reproduce my baseline problem of shared storage dumpicates...

  

DataGeek

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Jan 17, 2024 Jan 17, 2024

Copy link to clipboard

Copied

If you are each importing images into the respective catalogs from the same device or card, there will undoubtedly be duplicate files created

 

greg, can you say more about your statement above? 

 

Let me try to wrap my brain around this for a moment..  

 

The key issue here is about detecting duplicate images, a single catalog can do this for its imported images.  Then it will alert me if dups are present (I have not been using that option on the input screen!), I should uncheck those images (using icon at top) to keep them from being imported again.  I am not sure why dups should be occurring if I am using "delete origonals"...  Yes, if I cheat the process and reimport (cant use sd card with iphone) using USB on another pse2x catalog instance, and did not previously delete those imported images, then pse will not alert me to dups (and if I had heeded the warning! ugh), and unchecked, then yes I would be reimporting (to nas), those images....   

 

DataGeek

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Jan 17, 2024 Jan 17, 2024

Copy link to clipboard

Copied

Yes, if I cheat the process and reimport (cant use sd card with iphone) using USB on another pse2x catalog instance, and did not previously delete those imported images, then pse will not alert me to dups (and if I had heeded the warning! ugh), and unchecked, then yes I would be reimporting (to nas), those images.... 

 

Sorry, meant to say at end above that yes, I would be creating duplicates if I did that...   but I still think something else is afoot, but thanks for the better understanding ...  and will comntinue here to reproduce... 

 

 

DataGeek

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Jan 18, 2024 Jan 18, 2024

Copy link to clipboard

Copied

Hi folks, here with an update, ran another import this am, and see an actual duplication.  Here's the artifacts:

1.  I took a couple photos on my iphone (Iphone 10, XsMax ios 17.2.1).  it showed 3 pix, 1 shared from wife and 2 taken with iphone.  I am calling that shared photo an artifact for now, I think IOS just shows that I have a picture that came into my msg que, it did not import using PSEOrg. Here is a screenshot of that iphone "photos" tab (two pix and the shared file reference = 3):

 

PhotoGeek2121_0-1705604240985.png

 

I then added 2 iphone "screen shots" to capture the iphone starting point (for this dialogue here).  So the "starting point" is 4 total pix.  I opened PSE23 org and selected camera/iphone, and the import advanced window shows as follows:

 

PhotoGeek2121_1-1705604976686.png

 

You will notice that PSE23Org import "sees" 8 files, the 4 origonal files have been duplicated.  The import completed (but had to close organizer and reopen it to complete as I have described above).  Because I export using "date taken" folders, 4 of the files went into the 1/17 folder, and 4 went into a 1/18 folder (on my shared NAS server). (here are pix of those two folders):

 

PhotoGeek2121_2-1705605427279.pngPhotoGeek2121_3-1705605571786.png

So, documented above is a specific example of my issue with duplication during PSE org import.  These dup files appear identical in size, type, just have the 4 digit number accrued.     

 

Help!!!    

 

 

DataGeek

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Jan 18, 2024 Jan 18, 2024

Copy link to clipboard

Copied

@PhotoGeek2121 said:

If you are each importing images into the respective catalogs from the same device or card, there will undoubtedly be duplicate files created

greg, can you say more about your statement above? 

 

When you import files into a catalog using the APD (Adobe Photo Downloader), you are copying files from the phone or memory card, or USB drive to your computer's hard drive (which, in your case, is the NAS drive).  You have chosen to store your files in subfolders of Date Taken.  (I do the same.)  Once the files are copied to the hard drive, the files are imported into the catalog and their stored name and location are placed in the catalog's database.

 

If you use the Organizer on one computer to import files from a device (e.g. your phone), they will be copied to the hard drive and will (depending on the options you choose) have a name such as Img0001.jpg.   That name will now appear in the catalog.  If you now try to import the same photo into the Organizer's catalog on the second computer, using the APD, the file will now be copied again to the chosen Date Taken folder.  However, the OS will not allow two files with the same name to be located in the same folder.  So, the file will be copied to the folder but be given a different name which will usually have a numeric suffix such as Img0001-1.jpg.  Since the computers are using different versions of Elements, they are using different catalogs.  So, when the second catalog attempts to import the same photo from the same device, it will not be recognized as already existing in the catalog, and the copy will be imported (and the APD will not filter it out as a duplicate).

 

If you have your NAS set up as a watched folder for each catalog, the first computer will import the -1 copy and the second computer may import the original.  But because these are different (but duplicate copies of the) files, both may now appear in each catalog.

 

Now, having said all of that, you may be having a separate duplication issue with photos imported from a camera memory card.  I have only experienced this problem with my iPhones.  To determine whether this is the case, connect your iPhone via the APD and click on the Select Folders icon.  If you open up the folder hierarchy, you may find that you have two DCIM folders.  If they are both checked for import, this will cause duplicate files to be copied to your computer and imported (as separate files).  Uncheck one of the DCIM folders.

 

Ashampoo_Snap_Tuesday, January 9, 2024_0h6m17s_multi.png

There may be other possibilities for duplication, particularly given your comments about sharing photos over the iPhone with your wife.  When you share a photo from one iPhone to another, depending on what apps you use, you can end up with duplicate photos in your Photo Library.  However, again depending on the app and how you have saved the shared photo, you may or may not have the shared photo in the folder system from which Elements imports the photos.

 

A few final thoughts:

  • I have never used the After Copying Delete Originals option.  For me, that is fraught with danger and I see no benefit to using this approach, particularly given some of the problems you have described with importing.
  • I believe that the problems you are experiencing with the import process completing only after you relaunch the program is because you are using a NAS which creates performance issues and is probably one of the reasons why Adobe does not recommend using this storage method.
  • As I understand, some of your follow up posts, you were not able to reproduce the duplication problem under some scenarios.  That suggests to me that the program is working correctly and this problem has been caused by some unusual workflows that you are using with the two computers (and two phones).
  • To further investigate what is happening with your import procedure,  use the Sort by Import Batch option to see what and when each pair of duplicates was imported.
  • Also investigate the file location of each duplicate by clicking on the file location in the Organizer's Info panel on the right.

Hope this helps you to sort out the issue.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Jan 19, 2024 Jan 19, 2024

Copy link to clipboard

Copied

greg, WOW!  your a rockstar for the added detail on what is going on behind the curtain...   i'll parse and respond as soon as I can...  TY!!!

DataGeek

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Jan 19, 2024 Jan 19, 2024

Copy link to clipboard

Copied

LATEST

@PhotoGeek2121 said:

 

Hi folks, with Gregs help I think we have figured out what is going on and how to solve it.  I have copied Gregs last msg above (text italicized and underlined below) and replied to each paragraph (my text not underlined). 

 

If you are each importing images into the respective catalogs from the same device or card, there will undoubtedly be duplicate files created

 

greg, can you say more about your statement above?

 

When you import files into a catalog using the APD (Adobe Photo Downloader), you are copying files from the phone or memory card, or USB drive to your computer's hard drive (which, in your case, is the NAS drive).  You have chosen to store your files in subfolders of Date Taken.  (I do the same.)  Once the files are copied to the hard drive, the files are imported into the catalog and their stored name and location are placed in the catalog's database.

 

Ok, this helps explain the “Two Step Import” see.  The first step starts quickly after “get media”,  then completes (copies images to NAS and deletes from iphone), then I see a pause (and sometimes I must restart APD Organizer) then see the images imported into APD Catalog Database.  It’s odd that the delay I am seeing starting step 2 is the APD catalog DB update (on local hard drive) and not trying to access my local networked NAS (step 1)?   (more below on this topic).

 

If you use the Organizer on one computer to import files from a device (e.g. your phone), they will be copied to the hard drive and will (depending on the options you choose) have a name such as Img0001.jpg.   That name will now appear in the catalog.  If you now try to import the same photo into the Organizer's catalog on the second computer, using the APD, the file will now be copied again to the chosen Date Taken folder.  However, the OS will not allow two files with the same name to be located in the same folder.  So, the file will be copied to the folder but be given a different name which will usually have a numeric suffix such as Img0001-1.jpg.  Since the computers are using different versions of Elements, they are using different catalogs.  So, when the second catalog attempts to import the same photo from the same device, it will not be recognized as already existing in the catalog, and the copy will be imported (and the APD will not filter it out as a duplicate).

 

I am using the APD provided post amble 4 digit number that advances over time if the base name (date)? is the same:

              

PhotoGeek2121_0-1705734213091.png

 

I rarely see a “-1” file name due to a windows duplicate. I also rarely (if ever) will upload a successful APD import using my second computer, if it was successful on computer 1, why would I run again on computer 2?  Its only now that computer 1 APD has stopped working that I have attempted to upload the images on that iphone using computer 2.  In this case those files have never been uploaded and are not on my NAS folders.  But if I had I agree that this scenario could cause duplicates.  

Again, my strict workflow is to keep iphone 1 and computer 1 separate from iphone 2, Ipad2, SD cards, and computer 2.  Separate APD’s on each (pse22 on computer 1, pse23 on computer 2).  Local catalogs, but shared NAS “Repository” with folders by year/date.  Over 100K files. I am certainly open to input on a better model.  I have seen some threads here that seem to recommend use a 3rd party app/tool to upload/import, then use the APD file/folder tool to “import” into organizer and then to “manage” the database from there?  Does this make more sense?  APD seems pretty intuitive, I am not sure the extended family can grasp more 3rd party apps and interfaces, and workflow. 

 

If you have your NAS set up as a watched folder for each catalog, the first computer will import the -1 copy and the second computer may import the original.  But because these are different (but duplicate copies of the) files, both may now appear in each catalog.

Now, having said all of that, you may be having a separate duplication issue with photos imported from a camera memory card.  I have only experienced this problem with my iPhones.  To determine whether this is the case, connect your iPhone via the APD and click on the Select Folders icon.  If you open up the folder hierarchy, you may find that you have two DCIM folders.  If they are both checked for import, this will cause duplicate files to be copied to your computer and imported (as separate files).  Uncheck one of the DCIM folders.

PhotoGeek2121_1-1705734213105.png

Greg, you are so correct!!!!!

At first I only see one apple folders to pull from, but think this is what you will see before you “trust the computer from the iphone:

PhotoGeek2121_2-1705734213106.png

 

 

After a couple of tries to select and view this “media folder to pull from” this view evolved to:

PhotoGeek2121_3-1705734213116.png

 

Notice that my 6 pix have been duplicated to 12, and the iphone sub folders have expanded to the structure seen above.  Further fully expanding to the “down carats” shows this structure where in fact I can select two individual folders.

PhotoGeek2121_4-1705734213118.png

 

Deselecting either folder changes the designated import targets to just the baseline 6 unique Pix!!!!!!

I was not aware of this (Apple?) unique nuance where my baseline images are structured within 2 file structures.  Nor was I aware that APD can make this visible (and selectable) using the folder icon in the (advanced?) import window.  Ty Professor Greg for lighting the path of understanding!!

I imported after “unchecking” the 202401_ source, both steps finished back to back with barely a hint of delay…   6 files only.  “Priceless”!!!!  The hangs/delays seem related to this dual source file artifact.     

 

There may be other possibilities for duplication, particularly given your comments about sharing photos over the iPhone with your wife.  When you share a photo from one iPhone to another, depending on what apps you use, you can end up with duplicate photos in your Photo Library.  However, again depending on the app and how you have saved the shared photo, you may or may not have the shared photo in the folder system from which Elements imports the photos.

 

A few final thoughts:

 

I have never used the After Copying Delete Originals option.  For me, that is fraught with danger and I see no benefit to using this approach, particularly given some of the problems you have described with importing.

 

I find that my iphones become littered with Junk that cannot be uploaded (live photos, slomotion, advanced files formats like HEIC, HEIF, etc).  It’s a constant battle to keep my “photo” icon clean.  Deleting helps that, I also was concerned about dups…   it also helps to keep my media sources clean, if its empty, its been uploaded…     

 

I believe that the problems you are experiencing with the import process completing only after you relaunch the program is because you are using a NAS which creates performance issues and is probably one of the reasons why Adobe does not recommend using this storage method.

 

This relaunch artifact seems directly related to the multiple folder sources on my iphone, when I clear those sources, the APD import runs flawless.  When I use APD to import from a SD card it runs flawless… both are using my local NAS repository (and gigabit network appliances).  Yes I can see windows file access pauses once in a while, but APD seems to have my Synology NAS on speed dial.    

 

As I understand, some of your follow up posts, you were not able to reproduce the duplication problem under some scenarios.  That suggests to me that the program is working correctly and this problem has been caused by some unusual workflows that you are using with the two computers (and two phones).

 

Yes, hidden Apple “feature” that requires some hand holding to mitigate!!

 

To further investigate what is happening with your import procedure,  use the Sort by Import Batch option to see what and when each pair of duplicates was imported.

 

The dups can be seen on the APD input screen, when you have a large number of files being imported it is easy to miss.  Agree that there are likely more root causes to the “duplicate” image “universe”…  but for now lets call this a win and close this thread, Ty Greg!!!!   

 

Also investigate the file location of each duplicate by clicking on the file location in the Organizer's Info panel on the right.

Hope this helps you to sort out the issue.

DataGeek

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines