Skip to main content
Known Participant
February 2, 2018

P:a No way to be sure if photos are in the catalog. zero.

  • February 2, 2018
  • 16 replies
  • 405 views

This post contains a video, for a full easy explanation of my situation. 
Video
https://www.youtube.com/watch?v=sThVzs0Q5pc&feature=youtu.be

The Issue in a Nutshell

You might say Import does not detect duplicate photos. So you have no way to point at various folders and be sure you imported that data. So I have no easy way to find out if my master catalog contains all my various drives (that I am sorting out) data.

I discovered this after doing a catalog import, I then did a standard import, not clicking import, pointing it at the photos folder of the catalog I just catalog imported, just to check if catalog import had worked. I selecting 'Don't import suspected duplicates', expecting to see zero or only a handful of orphaned files caused by incorrect deletion. It sees everything, every photo in the catalog! 

The main things I tried which I think relevant to say...

  • Since pointing it at the original /photos location on the other catalog drive did not work... I copied it to the same drive = no difference. (Some people talk about external drives causing issue, it ruled that out)
  • Pointed it at its own /photos directory = no new photos detected
  • Pointed it at its own /photos/folders I copied out to the desktop = new photos incorrectly detected.
  • Create a new empty catalog and did a catalog import of the mothership catalog, repeat = same result in all cases of above tests.
  • Try with one single folder of just 78 images = the same
  • Tried on another laptop with the same version of Adobe to rule out virus, bug, software, laptop, etc... = no difference, it's not my kit.

History
I started with a detailed Community Forum Post: https://forums.adobe.com/thread/2443613 and then since I discovered Adobe don't bother checking those, I then moved to a support case. Then I was asked to close my Adobe Case Number ADB-928182-W8X1 and post it here as a bug. After spending about 3 days working on this problem the lovely people at Adobe really touched my heart by giving me a one month credit for the Photography plan. How totally touching, moving, rewarding, understanding of them. No bug (or do they call it an enhancement request these days) provided, I feel like posting it here is a just a place for customers to vent, nothing more.

I think catalog systems are a terrible way to design software. I like how Exposure uses single tiny files in the images folder, as it means multi-users can also work on the same data together. Maybe it's time to move off Lr.

Version
I am using most up to data Lr and stuff, Lr Classic CC 7.1

This topic has been closed for replies.

16 replies

Jim Wilde
Community Expert
Community Expert
February 4, 2018
Yes, I guessed that would be the case. I finally got some memory cells working and recalled the issue that I had seen (and reported to Adobe) several years ago, and tested it again (in much the same way as you did) to verify my memory.

Basically, what happens is that, as you may know, Lightroom's duplication checking is based on three criteria....file-name, capture date, and file length. When you import images these three bits of information are stored in a field (called the Import Hash) within one of the catalog database tables....and it's that hash that's subsequently used to compare against new imports. 

Generally that all works well, except that when you do the "Import from Another Catalog" the contents of the Import Hash for all the images being imported from the other catalog are not brought in, i.e. after the import from the catalog, the Import Hash is blank for all the imported images. Thus when you subsequently attempt to import those same images from another source, the duplicates checking fails because there's no Import Hash to check against. I have a feeling there are some other instances as well where something similar can happen, but will definitely need to trawl through some old emails and/or forum posts to confirm that.

I hope that's at least explained what you have been seeing. I'll contact Adobe again about this.....it may be a bit of an edge case, but it's still something that should be avoidable.  
Known Participant
February 4, 2018
 Hi Jim, it's not file renaming, the examples I am looking at are nearly all not renamed. But your last question is a very good question...I think you might be on to something, I guess you're thinking.... 'Maybe the import doesn't detect duplicates if they were previously imported as a catalog import',  or 'maybe the import is changing the file meta in someway incorrectly and so if you point at the source again it doesn't recognise duplicates'...

THIS IS A BUG - HERE'S THE RESULT.
It's specific to data that has previously been catalog imported, even with a brand new empty catalog running tests today I can recreate it every time. Jumpy to the summary at the end of this post and read the 4 tests.

BACKGROUND FULL WORKFLOW AND PROBLEM IF YOU NEED
But just jump to the 4 tests at the end....

TEST 1 WORKFLOW
Firstly, like before, I noted a folder (a different one this time 20170219 UK ( I actually did it 3 times with 3 folders all the same results) that was imported in part of the previous catalog import, so it's in the new MASTER and the old TRAVEL catalog. I also checked the catalog is on the disk and in the catalog tree, and looks correct. The no. all match. Please note, these photos first arrived into the master catalog by a catalog import.

1. Point import within the MASTER catalog at its own 20170219 UK folder in the drives /photos folder, where the data should be found, and duplicates were detected correctly. It should see the data is already in the catalog.
2. Copy that same folder above (in the Master catalog photo folder) to the desktop (copy and paste) and point import at that, this time duplicates were incorrectly NOT detected, but they are in the catalog. It thought the images were new. The only change was copying and pasting it outside the directory the catalog points at. To confirm, this folder was previously brought into the catalog as part of the catalog import. I also copied it to the external drive or other locations, all the same result, it thinks they are new.
3. I then pointed import at the folder in the original location and duplicates were incorrectly NOT detected, it seems them as new, but they are in the catalog.

x3 repeats on different folders, all the same.

-----
JUMP HERE FOR 3 KEY TESTS AND RESULTS
------

TEST 1 - TEST ON DATA IMPORTED TO MASTER ORIGINALLY VIA CATALOG IMPORT
Just moving a folder that has previously been in a 'catalog import' in Lightroom Classic CC to a new location and point import at that, and it will not detect the files as duplicates and still want to import them. Hence you can't be sure if all your data is in by pointing at old drives or old directories. 

TEST 2 - CREATE NEW CATALOG > CATALOG IMPORT 1 FOLDER > REPEAT ABOVE
So I created a brand new catalog, brand new photos folder, did a catalog import selecting the one folder 20170219 UK. Then I repeated the above, and same result, it fails to detected the duplicates it should.

TEST 3 - CREATE NEW CATALOG > PHOTO IMPORT 1 FOLDER  > REPEAT ABOVE
So I created a brand new catalog, brand new photos folder, did a standard photo import rather than the initial catalog import, and then pointed at the other folder, it detects duplicates. So we have it, it's specific to the data that has arrived via a catalog import, not a photo import. 

TEST 4 - CREATE NEW CATALOG > PHOTO IMPORT 1 FOLDER > CATALOG IMP 1 FOLDER > REPEAT ABOVE
So I take it one step forward, create a new catalog, import one photo import as test 3 but this time then move on to do an additional import, this time a  catalog import of 1 folder. Everything was the same as test 3 but it failed to detect duplicates on the folder I imported via catalog import. So that's the problem, data that has been catalog imported previously is always seen as new data when doing a photo import.

SUMMARY OF BUG

Create a new catalog, do a catalog import of even just one folder, copy that same folder to any other location, try a photo import of it, and its contents won't be detected as duplicates, but it should be. If you did that same workflow but with a catalog that had an initial photo import, not a catalog import, and there is no problem. It's specific to data that has been catalog imported.

STEPS TO RECREATE

1. Create New Catalog on Lr Classic CC
2. Import a folder of photos with PHOTO IMPORT
3. Try to import that again with PHOTO IMPORT and you'll see duplicates correctly detected.
4. CATALOG IMPORT one more different folder to that same CATALOG.
5. Try to import that again using PHOTO IMPORT your see duplicates incorrectly detected. So you have no way when sorting out data and checking drives to know if your data is in the catalog.


Jim Wilde
Community Expert
Community Expert
February 3, 2018
I'm having the same memory issues, Bob.

I have to say that I've tried several times to recreate the issue that you're seeing, Andy, but so far without success. Every test I've tried so far, pointing the catalog to a copy of already cataloged images results in them all being correctly detected as duplicates (though that is NOT an instant process, it can take a while for the dupes checking process to complete....but that's seconds not minutes or hours). But maybe I'm not testing in the right way, and this is where my failing memory comes into play. I know that I've seen something similar in the past involving the Import from Another Catalog routine, but I can't remember the full details.

One question: does this problem (of pointing to another copy and not having them being detected as duplicates) happen with ANY folder, or only on folders that have been involved in a catalog Export or Import procedure? In other words, if you just import a folder of images from a card into Lightroom, close that catalog, copy that folder to the Desktop, restart Lightroom using the same catalog, then try to import that copy folder from the Desktop, does duplicates detection work correctly or not? 
Inspiring
February 3, 2018

I seem to remember someone saying that LR compares the filenames of new files with those in its catalog. So if you have changed the name of the file in the catalog, it will think the same file with its original name is new. Should be easy to try. I may be wrong; my memory is not so good these days!


Bob Frost

Known Participant
February 3, 2018
Hi Bob, I never rename or touch the RAW files before import from card, merge, or anything, I only deal with RAW. I let Lightroom import and manage them from the card. The only rename that is sometimes done is within Lightroom once a photo is ready to publish, at that point IMG_1234 might become a better clearer name for sharing and posting online. I rename in Lightroom using the Library tab and within the side panel. In terms of directory structure, again I never touch this except on occasion I might rename a few folders inside the Library folder tree within lightroom to have a more meaningful name on the end, e.g. YYYYMMDD turns to YYYYMMDD D.R. Congo. But when I compared data not seen as duplicate it was not a folder that had a meaning full name added, nor was it a folder that had any files renamed inside it, just 20170226.
Inspiring
February 3, 2018

Have you renamed all your files already on your computer?

Bob Frost