P: (Windows) Panel is limited to 1600 items (folders, keywords, etc)

[FeedbackCommunityMember]

Lightroom bug - hit a folder limit and unable to select the newly imported folder.452497 photographs across many drives in one catalog.The S drive has 214341 photos. It's the largest drive and so had many more folders than the previous drives.S:\Master Photos\ contains ~1633 folders. I know that's a lot but that is how her filing system was started. Those are the sequential numbers from 2220 - 3853 but there may be a sub folder or two or deleted folder in there so that's why the number is ~ approximate.Folders are labeled "sequential number"-"date" so 3853-20150316Adding one more folder caused Lightroom 4 to not allow the last folder to be clicked on and not show any images. The T:\ drive (offline, collapsed and below the S:\ drive) was collapsed and the label was overlaid on top of the "Collections" section label. Updated to latest version of Lightroom 5 via her new cloud account, had it rebuild the catalog and the same problem occurred.Created a sub folder S:\Master Photos\2012 and moved 15 of the folders into that folder. The 2012 folder was expanded so the list was actually one row larger and the last good folder wouldn't open. I collapsed the 2012 folder and now all the bottom folders were accessible.It appears there is an object limitation or a fixed array or something that is causing the folder list to become finite. She said a similar thing happens on her keyword list until she collapses some of the groups but I didn't check that out.

[ssprengel]

This may be an operating system or programming language limit not something LR is doing different than how it was designed, which is what the term "bug" normally means. Any system, physical, or virtual, is going to have some design limits, and the only question is whether those limits will be encountered when using the system. The elevator in the building where I work has, maybe, a limit of 2000 pounds or is 6x6 feet square or whatever the actual limits are, so if too many people want to use the elevator at once, the limits may be exceeded and the solution is to split the number of people into smaller groups and use multiple elevator trips.

If I'm understanding what you're describing, it seems there is some sort of display-items limit, not an actual folder limit since collapsing a folder will let you see more, further down on the list.

I would suggest a simple reorganization of the physical folder structure, either breaking it up under 2 parent folders, a 2000 and a 3000, then move all the 2xxx folders under the 2000 and the 3xxx folders under the 3000 which should double the capacity but would require having to scroll down through 1000 folders with the other one collapsed, which still might be too many, so perhaps a more useful set of parent folders would be ones that have the year-month followed by the first sequence number by that month, so for your example folder, 3853-20150316, the parent folder that would contain it an all the other folders for March 2015 would be 2015-03-3nnn where 3nnn is not the literal folder name, but the lowest sequence number of the earliest March 2015 folder.

The parent folders would have both the month and first sequence number for the month so it'd be easy to locate the correct parent folder when either the date or sequence number is known. I have chosen to put the date part first instead of having the sequence number first, because the yyyy-mm is more of a regular pattern without any gaps so easier to scan down with your eye, and would also allow non-sequence-number subfolders inside if the organization is ever changed, it is unlikely that dates will never be part of the organization, so I put them first in the folder name structure. One more reason to put the date, first, is if you eventually find that older dates are rarely if ever needed, then another layer of parent folders can be made, but this time just yyyy-firstseq so an entire year's worth of subfolders can be collapsed at once instead of having to collapse individual months to get the list short enough to scroll to the bottom of.

There may be some other partitioning of subfolders in a few parent folders that makes sense depending on the subjects in those photos, but these ideas are what come to mind seeing that the existing folder names contain only a sequence number and date.
--
Basically I'm saying LR may not be able to accommodate the existing folder structure so the folder structure may need to be changed. Even if LR can be changed, it would take some months or years, perhaps, so what will you do in the mean time? Folder reorganization seems reasonable as a temporary if not permanent solution.

[johnrellis]

The Keyword List panel on Windows LR does indeed have the same limitation on about 1600 items, at which point it fails ungracefully. The problem does not occur on Mac. As Steve said, the workaround is to introduce hierarchical folders and keywords.

[Rikk Flohr: Photography]

Confirming what John and Steve have already said:

There is a vertical limit on how much data can be displayed within a panel. The limit doesn't affect the number of folders, just the number that can be displayed. As John has indicated, the workaround for now is to have a more hierarchical folder arrangement to mitigate the number of same-level folders.

[FeedbackCommunityMember]

Steve, thanks for the info. I'm already having her move into "Year" folders to keep the number of sub folders in the 300-900 range. She'll probably move to something like a Drobo or Synology system where all her images will show up on one drive so the "year" folders will have to be in place before that.

The report was more for the developers/documentation to be aware of the limitation as they probably didn't build at test case with 1600+ folders on an 8GB Windows 7 machine to see what happens. When that limit is hit, Lightroom eats up all available memory and doesn't gracefully close. Maybe a better error trap to make the user experience more pleasant, although I don't know what they would check for to trigger an error if there even is an error. Maybe some documentation on "possible system based limits" in the best practices section. Documentation is cheap. I know this is an edge case but it may help others if they hit this problem.

5TB drives are out and some photographers might not have a good folder structure.

If you would like to remove the word "bug" in the first line of the first post be my guest as this appears to be a system limitation.

John, thanks for the keyword comment. I wonder if it is the way Mac handles memory or your Mac has more memory than the Windows machine I was working with.

[johnrellis]

"The report was more for the developers/documentation to be aware of the limitation"

Adobe is already aware of the limitation on Windows -- it occurs much more oten with keywords than folders: https://helpx.adobe.com/lightroom/kb/.... There have been many reports of people bumping into the limit with keywords, but this is the first one with folders I've seen over many years.

The Mac and Windows implementations of LR have different implementations for the lower layers of the user interface, and it appears that the Windows implementation of a scrollable list of items has a limit of about 1600 items. It is sloppy programming that the limitation isn't handled more gracefully, much less that there is a limit at all. There's nothing inherent in Windows that imposes such a limitation.

[john beardsworth]

" There's nothing inherent in Windows that imposes such a limitation. "

I understood that the problem is because Windows has (had?) an approx 32000 pixel limit on the size of a single control, and that Adobe haven't coded the UI to work around it.

[Rikk Flohr: Photography]

That is my understanding as well, John.

[johnrellis]

It's true that a low-level Windows control can have maximum height and width of 2^15-1 pixels. But nothing stops an application from coding its own scrolling controls, using a low-level window whose height is less than the screen height (or using a library that has done this). Any Windows application that has to display a very large list has to do this (and most do). For example, Windows File Explorer can display lists of files that exceed 32K pixels. Even LR does so for Library grid view, which can far exceed 32K pixels in height.

[Daniel Wiercioch]

I've spent a lot of time adding keywords to my photos. Well over 200000 photos. Probably several thousand hierarchical keywords. When I look at my list of keywords in the keyword panel, the list is truncate -- that is, I see about 90% of them, but near the end, the keywords stop showing and I can't scroll any further. I know the keywords are still there, because I can use them in searches. I just can't see or select them any longer. This was not a problem in Lightroom 5.

[Daniel Wiercioch]

Thanks for merging my thread in here. When I collapse some of my hierarchies as suggested, that allows me to work around the problem for now. Hopefully, Adobe addresses the problem at some point.

[FeedbackCommunityMember]

We supply a patient photographing system and use Lightroom as the DAM. The majority of our clients on Windows hit the list limit for folders as each tethered session creates a new folder. We have a system for splitting into hierarchy through a plugin but it is inconvenient and it would be great if Adobe could fix this.

[johnrellis]

Every Windows application with large scrolling lists  has to cope with the maximum size that Windows imposes on low-level windows (32K pixels) by repainting the windows as necessary. Most applications do that by using commonly available libraries of user-interface controls that do the work for them.

But Adobe took a shortcut when they implemented LR Windows, choosing to implement their own controls but failing to complete the work. They did it for Library grid view but not Folders, Collections, Keyword List, and other panels. And they've never owned up to taking that shortcut.  See here for details: 
https://feedback.photoshop.com/photoshop_family/topics/bug-that-causes-collections-and-publish-servi...

[FrostyOfTheNorth]

I've run into something a few times with Lightroom (both older versions and now the latest version, Lightroom Classic CC 8.0) in which at times the folder list in the Library module cuts off and does not display the bottom folders.  When that happens, the scroll bar is present but the folder list ends prematurely, often with dozens of folders missing at the bottom.  That complicates things because my naming convention for photo shoots involves folder names that start with the date, so my latest shoots are always at the bottom.  The solution has been to restart Lightroom -- and if that doesn't work, to reboot the computer.  Eventually the folder view is complete again... until the next time.

I thought the issue might be an odd quirk localized to my computer but it happened again today as I'm migrating to a brand new computer.  Brand new install of Lightroom Classic 8.0 on a clean install of Windows 10 Pro, with my catalog transferred over.  I fired up Lightroom and right off the bat the folder view was cut off with a few dozen folders missing from the bottom of the list.  I restarted Lightroom several times, rebooted the computer several times, but the missing folders remained missing.

Over the last few months I developed a hunch that this issue is related to screen resolutions (I have a 4K laptop with two larger external monitors) so I tried changing the screen resolutions and scaling settings in Windows and... after trying several options I got the folder list to be complete again.


It seems to me that there's a bug in Lightroom that doesn't always account for resolution and scaling settings.  Anyone else run into this problem?  I know I can filter the folder view for a specific folder name (which I didn't think to try when folders were missing), but I'm hoping this bug gets identified and resolved.

[johnrellis]

The workaround to this long-standing bug is to introduce more hierarchy into your folder structure and keep most of the parent folders collapsed.

The underlying limitation is on the size of the low-level operating system window, so as you increase the actual font size, fewer lines of text can fit into the window.

[FrostyOfTheNorth]

If that's the case, then why does it work most of the time?  This does not appear to be a hard limit.  Like I mentioned in my original post, after messing with resolution and scaling the folders are back.

[johnrellis]

You could be close to the maximum. The experience with keywords is that the behavior gets flaky and unreliable "near" the maximum. If you doubled the number of folders, you'd always encounter the problem.

[FeedbackCommunityMember]

I am another user who has hit this limit within the past few months, and only just been made aware of the underlying cause. It seems to me that reaching ~1500 folders in a single parent folder is very likely, given that one of the import options is "organize by date" with a date based folder naming convention. In my case this was after around 8 years, with around 1600 automatically named import folders (\pictures\import\yyyy-mm-dd). Of course it's possible to work around the problem by rearranging the folder hierarchy so that import folders are (for example) in a year based parent folder, but that's a manual process, not a standard import naming convention.

[FrostyOfTheNorth]

I had to add a year-based parent folder to solve the problem.  It was a painful and otherwise unnecessary process.   I wish Adobe would have addressed these issues instead.  I don't know if they have since I posted about about.

[mastix]

This is a totally flawed component of Lightroom . It does not work correctly. Once you have many keywords, the program collapses. You cannot add anymore without seeing them. Trying to make a hierarchy might seem as a good solution, but it makes things worse, as then you have to deal with duplicate keywords in and out of the hierarchies. The List also becomes totally unresponsive once you have many keywords . You are not more allowed to drag and drop one or many keywords to a top hierarchy keyword if you have too many on the list and go to the lasts ones (something about a windows limit).

You can solve this creating "artificial" hierarchies like alphabet letters. But that brings a number of new problems. When you keyword and image. I use a Lightroom keyword plugin all those new keywords get saved outside the hierarchy. But you don't want to have them that way as if the lists becomes to long it will hit again the unresponsive problem because too many keywords. But now you have sometimes the same keyword inside and outside the hierarchy (if it is a new keyword no problem, but if it is one you have used before and put inside the letter you are on a road for headaches).

You cannot simply select a group of all letters with A now and throw them inside the A hierarchy because once Lightroom detects one of those duplicated you can not throw them all inside.

So now you have to go one by one and leave the duplicated one for later to deal with. Once you have done this (labor and time consuming) you now have to search for every duplicate one that is left out select the images , throw those images inside the duplicated keyword inside the hierarchy and delete the one outside the hierarchy ( DO NOT delete before you assign them to the one inside the hierarchy or those keywords will be deleted for those files).

All this could have been fixed by Adobe just letting us merge duplicate keywords or give us a click option do NOT create duplicate keywords and assign them wherever they where created first.

For me, this is by far the WORST implementation in Lightroom classic, and it gets not fixed over the years. Adobe is just not listening to customers here.

In my case, I don't recommend to anyone Lightroom as a Dam solution. For developing and retouching, yes. For classifying photos there are at the moment much better , advanced and above all responsive solutions.