P: Won't import a file with large numbers of images in "import from another catalog"

[kmedig]

I'm breaking up my master catalog into individual years so they're smaller and run faster. I created a new catalog (say 2019) then choose "import from another catalog", which I choose my master. It takes a while, then compiles a list of files to import (they are originally all checked boxes). I uncheck all the boxes, then select only files from 2019...this works really well...until I get to a file that has a large number of images (say around 8000 or higher). It refuses to allow a checked box on these large files (even though they were originally checked). Seems to be a bug, I've tried everything to work around this. Any suggestions?

[Rikk Flohr: Photography]

I am not seeing this.  Just to be clear: You have a single folder with 8 K images or your cumulative check marks go up to 8K?

[kmedig]

Hi Rikk, thanks for the quick reply. I just tried to import from my master to a new catalog and ran through to see which would and wouldn't import. I had a file that was 6409 images that allowed it to be checked (see attached...all ones smaller than this were good to go) but one with 7468 images (and all that were higher) weren't able to be checked. 

[Just Shoot Me]

The SIZE, Number of images, in any catalog has no barring on how fast LR runs.
What you are doing is Counter Productive to using LR the way it was designed to work.
To divide your catalog, separate your images, into years you should use Folders for each year. Not a different catalog for each year.

[kmedig]

Thanks for your input on this, it's something I'll definitely consider but it doesn't address the problem I found.

[Just Shoot Me]

Your screen shot show that the catalog you are trying to Import TO already has images in it. I thought you said you created a New Catalog and tried to import your original, working, catalog into it.
Also the SS show you do not have any scroll bars on the import from another catalog dialog window. That could be a problem with the OS and not LR.

When was the last time you restarted your Mac?

Normally when calling up the Import from another catalog option once you select the catalog to import from when the dialog box comes up "All Folders" is checked along with ALL the individual folders listed below it.

Can you Scroll through the list of folders using a mouse scroll wheel or the scrolling action of the Trackpad?

Also you keep saying "A File" contains XXX number of images. No File contains more than one image. Unless it is a ZIP file and or some other type of compressed folder that has multiple files inside of it.
It is FOLDERS that can and do have multiple FILES in it.

[kmedig]

Before these screenshots were taken, these are the steps I took:

-created a new catalog, let's call it 2019 (absolutely empty catalog nothing in it).
-with this new catalog open, I go under FILE and "import from another catalog"
-I choose my huge master catalog with everything in it. 
-Lightroom takes time to create the list of FOLDERS to import. In comes the list you see in the screen shots, everything from my master catalog.
-However they all come pre-checked because, as you stated, there's nothing in the new catalog as duplicates.
-I uncheck them all (at the top of the list) and then go through and CHECK ONLY THE FOLDERS I WANT IN THE NEW CATALOG.
-any of the folders with under (in my case) 6668 images comes across no problem. Anything over 7468 images doesn't.
-if I wanted everything to come over, it would have fine, but if I UNCHECK AT TOP AND SELECT SPECIFIC FOLDERS only ones under a certain amount come into the new catalog, anything over a certain amount don't.

I'd love you to try it on your system following these steps. Create a fresh new catalog, import from another catalog and try to import a single, not all, but a single folder with say 10,000 images in it...would love to see if it works! (ps. sorry my nomenclature on folder/file was confusing originally...)

[Just Shoot Me]

I have tried it and it is working as expected, IE Properly. Although none of my FOLDERS, Please STOP using the word File when the word FOLDER should be used, have that many images in them. Even my By Year FOLDERS only contain, at the most, 4-5 thousand image FILES in them and those are broken up into Subfolders by day/month dates like 0523.

Again when was the last time you restarted your Mac. Are all the external drives you have images on connected to your system. If the import process can't locate the actual FILES in the FOLDERS and drives they are supposed to be in the import process may exclude them.

Also you keep saying you created a NEW BLANK catalog but the screen shots you posted clearly show that whatever catalog you are importing TO has images in it.


So I don't know to what catalog you think your are importing TO but it is not a New Blank catalog

[kmedig]

Thanks for checking and yes, apologies for the imprecise nomenclature originally (I just edited it to fix the file/folder thing...I'll try to fix the title of this post too). 

Please don't get too hung up on the screenshot. I was just trying to show that I couldn't 'check' the box for the folder with the large number of images (the folder wasn't already in the catalog) and could 'check' the box with a folder under a certain amount. That screenshot was taken of the catalog I created for 2019 where I had already imported all the folders that would come over under the number of images/folder that seems to be the crux.

Honestly, the crux of the problem isn't bringing the folders over to an existing catalog from another (I've been doing it a tonne...I just did it a minute ago with select folders from my Lightroom catalog from my laptop...but they all contained a low number of images (under 4000 images) and it worked perfectly). 

It's like there's a bug where Lightroom won't allow SPECIFIC FOLDER SELECTION with folders containing a large number of images (I'm seeing this with folders with  ~7000+ images to be moved from one catalog to another). 

I urge you to try it by creating a folder with a large number of images (say 10,000), then create a new catalog and see if you can selectively pull it over from one catalog to the next (it's critical you 'uncheck' all the boxes then try to reselect the big folder). I'd love to hear if it works!

Either way, I appreciate the time you're taking to figure this out. (I shut down my computer every night etc).

[Rikk Flohr: Photography]

While I was able to reproduce on a folder of 13,000 images it is a very narrow use case. I went through dozens of test catalogs to find I had no test catalog with more than a couple thousand images in anyone folder (time lapses). I finally had to build a special catalog to break it. 

6000+ Images in a single folder is rare.
Fragmenting an Import from another catalog is relatively rare. 
Both combined are extremely rare. 

I would call it a bug.

If anyone else experiences this, hit the [Me Too] Button at the top of the page so we can gauge the overall customer impact. 

[Just Shoot Me]

Thanks for the tests Rikk. 
I suggest to the OP to not place so many images into one folder. 

[kmedig]

Thanks for confirming the bug Rikk and thanks for your time looking into it as well JustShotMe.

Hopefully it's something that the programmers can fix easily in one of the next updates to help photographers out there who shoot higher volumes of images.
  
Thanks again to you both and all the best.

[Rikk Flohr: Photography]

Version 9.3, an update to Lightroom Classic, went live on June 15, 2020 and contains a fix for this issue. Please install the update and respond back if it does not cure the issue for you. Thank you for your patience!