Importing a "non-duplicate" duplicate

[kenf51]

Hello everyone

I have a MacBook Pro with M1 chip running Sonoma 14.4.1

I have Adobe Elements Organizer 2023 version 21.0.0

I do genealogy and am scanning old photo negatives.  A few years ago I had an inexpensive scanner and scanned a lot of my photo prints and imported them into PSE.  I now have a better scanner that can scan negatives so I am re-scanning the negatives of many of the same photo prints only at a much higher resolution than I had used before.  (With much better results I might add).  My problem is that when I import these newly scanned files (all with different date, different time, different filename, different size)  some of them only show a placeholder and I get a message something to the effect of "The file already exists in the catalog".   I did a visual search and can find the offending "duplicate" file, it is the same "photo" that was scanned years ago but at the lower resolution.  It would seem that PSE is recognizing the face of a named person or other elements in the photo and rejecting the newer, higher resolution version of the same photo.  I will ultimately be deleting the old versions of the photos but they have meta data (face names, captions, etc) that I want to copy to the new version before I delete them.  Is there some way I can have both versions of the photo in the same cataogue at the same time so I can manually transfer the meta data from within PSE?

Thank you,

-Ken

I

[Greg_S.]

It is very odd that Elements is refusing to import the duplicate files.  Can you tell us about how you are naming the duplicate files and where you are storing them.  The method that Elements uses to avoid duplicates has nothing to do with face recognition.   That process doesn't even begin until after the file has been imported into the catalog.

[kenf51]

OK, it’s a somewhat long description but here is what I am doing:

All my PSE database files are inside the sub-folder “PSE 2023 catalog 10-16-23” which is in my “Pictures” folder on my MAC hard disk.

I have been scanning the new negatives into a different sub-folder on the hard disk which is NOT inside my “Pictures” folder.  All the new scan files have the same filename except for a unique sequential 3 digit number included at the end of the name of each photo.  It is a long descriptive filename which I just made up and I KNOW there are no other files with that name already inside my PSE database.  There are other photos with the same 3 digit sequential number contained in their filename but not with the rest of the filename.  This is an example of the filename: 

“Med format negatives070.jpg” There are 136 files in the folder beginning with “Med format negatives001” and ending with “Med format negatives136”

With my cursor active on the “Pictures” folder in the tree structure visible from inside PSE I hit “Import from files and folders” and then select the folder which has the scans I just made and hit import.  They come in to PSE and now show up on the tree in the correct location (NOT inside the “Pictures” folder) but only about half or so have thumbnails, the rest have a placeholder icon with a torn page looking picture which I assume means a broken link or some other kind of error.  But at import I saw the message that some files are skipped because they are already in the database.  I ran a “reconnect all missing files” but it didn’t find any.  I then removed the folder from the database and then repeated the import with the same result.

[Greg_S.]

Ken, I know almost nothing about Macs, but if you used the same storage scheme on Windows, I would tell you to immediately make some changes.  You should not have the catalog folder in the same parent folder that stores your media.  I'm guessing this is not the Mac default location and that you have changed it manually.

I believe the full message that you saw when importing the scans says something to the effect that files were not imported because either they already exist OR they are not supported.  The torn page icon you describe is probably based on the fact that you are using high resolution scans and you may have exceeded the maximum size allowed by the Elements Organizer. So, I don't think it is a duplicate file issue. 

I am going to ping @Jeff Arola, one of our resident Mac Community Experts since I am not sure whether this maximum size applies to Mac machines.  Nor do I know if there is a Mac workaround.  (The linked article only applies to Windows machines.)  He can also provide advice about whether it is advisable to change the catalog location and how best to do it.

[kenf51]

I moved PSE (ands everything else) from Windows to MAC a couple of years ago.  I had some trouble with that transition but with the help of this community I got it worked out and it has been fine since.  The PSE folder locations, as far as I know, are where PSE put them when I installed it.

Path:

Root directory>Pictures>PSE catalog>Pictures,ProgramData,Videos

The folder “PSE Catalog” above is one I created to hold all PSE related files when I installed the program.  I am not sure where PSE stores the actual “Catalog” you are talking about.  I don’t even know exactly what the “catalog” is.  Is it one file? Is it multiple files?  Does it have a folder structure?  Can I see it in Mac Finder?  I don’t think I changed any default file locations when I installed PSE.

I did not see the “Not supported” message, only the “already exists in catalog” message.  As far as file size, they range from 5.4MB to 12.4MB.  The 12.4MB file came through ok, but many smaller ones did not.

[Monte307]

Would you mind uploading two of the files that PSE considers duplicates?  I have a Mac running Sonoma and would like to see if the same thing happens on a different machine.

[Jeff Arola]

Are the 3 digit numbers added with the scanner software or renamed later in Finder?

[kenf51]

I have an Epson V-600 scanner and I am using the "Scan 2" software. Yes the sequential numbers are added automatically by the Scan 2 program.

[Greg_S.]

@Jeff Arola does the maximum file size limit apply to Macs?  The HelpX article only talks about Windows and fixing it via the Registry.  As far as I can recall, the torn page icon described by the OP only appears for files that exceed the size requirements or unsupported formats such as raw files newer than the installed ACR plug-in.  The latter would not seem to apply here.

[Jeff Arola]

Yes, as far as i can tell, the same maximum file size limit applies on the Mac side, though i haven't been able to find any documentation on whether that can be changed like on Windows.

[kenf51]

It appears that it might be a maximum file size issue.  I read the article linked to earlier "Maximum Image Size Limits" that stated the maximum file size is 72 megapixels (75,497,472 pixels). I went through and calculated the megapixels for the first 16 images I tried to import.  (I rounded off the last 3 digits) The first 8 images did not go through and the smallest one of these was 76,350,000 pixels.  The next 6 images did import correctly and the largest one of them was 75,166,000 pixels.  Image 15 was 75,934,000 pixels and did not come through.  Image 16 did come through and was 74,046,000 pixels.  It would seem that the limit is somewhere between 75,934,000 pixels and 75,166,000 pixels, which agrees with the value given in the article.   I am disappointed in this situation.  I am scanning these old family negatives at 3200dpi so it takes almost 2 minutes for each one to complete and I have already scanned 454 images so as you can see this represents a lot of time!  I hope there is some way around this problem.

[Greg_S.]

You have a couple of options.  First, you should be able to open the file in the Editor.  You can then resize it there using the Image>Resize>Image Size command.  If you really need the full resolution of the original file, you can save the resized file with a new name and choose to save it in a version set with the original.  The smaller file will then be displayed in the Organizer grid with a visible thumbnail and the torn page icon will appear for the original file in the stack underneath.

You can also batch convert all of these scans, using the Editor's File>Process Multiple Files which allows you to resize all files in a selected folder by a certain percentage.   It sounds like you only need to reduce the file sizes by a small amount.

[kenf51 · Accepted solution]

Thanks for your suggestions Greg.  I did a little experimentation in the Editor and found I could either resize the photo a little or crop the photo a little and either one resulted in the photo thumbnail now showing up.  What I didn't expect was the reduction in file size.  Either method resulted in a reduction in file size of more than half although I could not see any difference in the image when zoomed way in.  What's going on?  

I will experiment with doing this in batches too as you suggest.  I would like to just crop the images and then overwrite the original but I can't seem to do that, PSE insists that I rename the edited file.  Even when I delete the original from the catalog it will not let me rename the edited photo with the same name as the original unedited photo.  Is there a way to do that?  It wants me to make "versions" but I don't really care about the original so would rather just overwrite it.

[kenf51]

I was a little hasty when I posted my previous reply.  I can get the thumbnails to show up by cropping the photos, but the cropping procedure makes the file size get very large.  I don't want to hijack this thread so I will not ask questions about it here.  My original question was answered and I thank all who participated.  

I would still like to know if there is a way to bypass the maximum file size limit on my Mac like there is on Windows machines.

[Monte307]

According to the above link “The Organizer in Photoshop Elements limits the maximum dimensions of an image based on the amount of RAM installed in your computer. “   So possibly having a Mac with more ram would fix the problem. Or maybe not.

I did some experimenting and it seems the precise file limit for the thumbnails to display (on my Mac with 8GB ram) is 75,500,000 (see attached files OK.jpg and notOK.jpg).   It would be interesting to see if the too-large file for an 8GB Mac would display ok on a Mac with 16GB ram.