Skip to main content
2k.Pandey
Known Participant
December 4, 2018
Answered

Lightroom Classic CC 8.0 large sized backups and LRCAT file

  • December 4, 2018
  • 7 replies
  • 7380 views

Hi Everyone!

Experiencing two weird issues with respect to LRCAT file size and LRCAT backup size.

1st problem is Catalog size is more or less independent of number of photos.  So 152 shot catalog and 25 shot catalog, are both 200 MBs unlike how it previously was. Here, I'm talking about the file size of the LRCAT file itself and nothing else. Just the catalog. As sometimes I've to export a 3 shot LRCAT file for the client when they already have the original RAW files and all I need to send them is the LRCAT file and nothing else.

2nd, compression ratio for LRCAT file (when compressed and saved by Lightroom) was around 80%. This was more or less similar to what you would achieve if you manually compressed an LRCAT file. While now theres barely 5-8% compression. Basically, 200 MB LRCAT would now be backed up as 180 MB compressed zip file. Earlier that was 10 MB or so. So all LRCAT backup folders are growing at a ginormous rate.

I hope that this is something that adobe can resolve and not something permanent with LR 8.0. As its making image transfer really difficult for people where good internet connection is still a luxury.

Is someone else experiencing this? Can someone from Adobe advise on when (or if) this will be fixed?

Thanks in advance for any help

This topic has been closed for replies.
Correct answer path72

Thank you for your reply.

simonsaith  wrote

Checked the catalog, seeing that the 3D color LUT took up the majority of the size of the new catalog. The catalog stores a total of 1021 entries of custom 3D LUT (not from Adobe), presumably coming from some custom camera profiles/presets.

Lightroom caches unique 3D LUT both in the catalog and on the disk to facilitate transportation of catalogs among different machines and quick access to them at develop time. Lightroom does not keep a copy of the 3D LUT at each history step.

The catalog size increase for a newly created/exported catalog is a manifestation of this. It was introduced as part of the Lightroom Classic 7.3.

Given that a reference to a 3D LUT could be buried in one of a history step (customer can go back in the history step to reference a 3D LUT at any time). Lightroom catalog does not currently have an infrastructure to do proper reference counting of custom 3D LUT usage. So it just includes a copy of every custom 3D LUT ever used.

We have to consider this case of multiple catalog usage scenario and see if we can optimize it. Thanks for the report.

P.S. The 3D LUT is also cached locally on disk at "~/Library/Application Support/Adobe/CameraRaw/Tables" on macOS. There is a similar Window directory location.

I checked the directory you mentioned (TABLES) and I found 1021 files I think which totalled up to 172 MBs which is also close to the resulting catalog size. So its definitely relating to that. Also, whats interesting is, even after deletion of these files, they're automatically created again in a second after any new catalog is created or opened on an already infected machine. I checked a non-infected machine and this directory for them, is totally empty. Deleting camera profiles hasn't worked either.

I am trying various permutations and combinations to alleviate the issue as two of my main machines are infected and work on them isn't possible because of this. I usually export a lot of sub-catalogs to send to my editors, and they're all super heavy even though they have very less images in them.


In order to clean catalogs presenting the issue, I've found the following workaround.

1. Make sure the folder

Application Support/Adobe/CameraRaw/Tables

is empty.

2. Within an SQLite editor, delete all the rows in the table

Adobe_libraryImageDevelop3DLUTColorTable

for each plagued catalog (delete all the rows of records, not the table itself).

3.

As size of .lrcat files is not reduced yet, open in LR each catalog, optimize it then close it.

In all my non-infected catalogs the table Adobe_libraryImageDevelop3DLUTColorTable is empty, therefore emptying this table outside LR should not produce any damage to the catalog itself.

7 replies

Participant
February 24, 2021

I am having the same issue. My catalogs are 520mb. Huge! I did that by deleting the Table folder and it is working if i open again one of the catalogs with 520 mb the folder is getting full again.

2k.Pandey
2k.PandeyAuthor
Known Participant
February 24, 2021

Hi! Sorry to hear that. The problem hasn't yet been solved by Adobe but the community here has since pointed to a reasonable workaround. Empty tables folder, Open each infected lrcat in a SQL editor and remove all entries inside of '3D colour table', save and exit. Then open lrcat and click optimize and exit. The size would have reduced back to normal and all other problems solved. Just take care to never open an infected catalog without repairing it first. 

jackc80362305
Participating Frequently
December 10, 2018

I tried using Notepad ++ to compare two of my catalogs today, one pre v8.0 and one post.

The difference was 171,000 lines of code in pre v.8 and 1.7 million in the second.. needless to say Notepad ++ Crashed in trying to open the v8.0 catalog and never made it far into the direct comparisons. With so many lines its near impossible to compare but I really hope this bug is rectified soon as the file size is a very frustrating bug.

ManiacJoe
Inspiring
December 12, 2018

Doing a file comparison on the two SQL data files is not helpful. It is possible to have the two files contain the same info but still be different at the byte level.

jackc80362305
Participating Frequently
December 9, 2018

I am also experiencing this over a couple of machines now. I have tried rolling back to older versions of LR CC but still having the same issue

Is there a fix for this or does one know where to alter the catalog file to change that soft proofiing profile (if that is the cause of the large file sizes). I deal with a lot of LR catalogs each day and this is crippling my ability to upload/download and share.

Thanks

Community Expert
December 8, 2018

Opened the catalog file linked above in a text editor and it appears that the 180MB is a gigantic color profile set as the default soft proof profile. It appears Lightroom stores this profile in the catalog file! Try changing the default proof profile to something simply like sRGB and then exporting a catalog.

Also if it hasn't been mentioned before submit a bug report to https://feedback.photoshop.com

jackc80362305
Participating Frequently
December 9, 2018

Hey Mate,

I checked my proof profile and it was still set to Adobe 1998.. same result changing it to sRGB. What were you looking for inside the catalog file with the text editor? Wondering if I can try change it this way, been trying everything and cant stop the file size from exploding on closing any new catalog.

Thanks!

Community Expert
December 10, 2018

I just opened the lrcat file in a text editor and right before a giant

block of data that goes until the end of the file there is a header that

seems to indicate a proofing profile is starting. I don’t think you can

easily edit this file but you might be able to manipulate it using MySQL

from the command line.

On Sun, Dec 9, 2018 at 2:22 PM jackc80362305 <forums_noreply@adobe.com>

NitinGupta7
Adobe Employee
Adobe Employee
December 8, 2018

2k.Pandey  wrote

Hi Everyone!

Experiencing two weird issues with respect to LRCAT file size and LRCAT backup size.

1st problem is Catalog size is more or less independent of number of photos.  So 152 shot catalog and 25 shot catalog, are both 200 MBs unlike how it previously was. Here, I'm talking about the file size of the LRCAT file itself and nothing else. Just the catalog. As sometimes I've to export a 3 shot LRCAT file for the client when they already have the original RAW files and all I need to send them is the LRCAT file and nothing else.

2nd, compression ratio for LRCAT file (when compressed and saved by Lightroom) was around 80%. This was more or less similar to what you would achieve if you manually compressed an LRCAT file. While now theres barely 5-8% compression. Basically, 200 MB LRCAT would now be backed up as 180 MB compressed zip file. Earlier that was 10 MB or so. So all LRCAT backup folders are growing at a ginormous rate.

I hope that this is something that adobe can resolve and not something permanent with LR 8.0. As its making image transfer really difficult for people where good internet connection is still a luxury.

Is someone else experiencing this? Can someone from Adobe advise on when (or if) this will be fixed?

Thanks in advance for any help

Thanks 2k.Pandey for getting this over a call with me and experimenting with the issue.

The issue seems like a bug, but it spreads to every LR which opens the infected file. I was able to record the video while the issue was being created in a .SWF file. attached!

https://www.dropbox.com/s/7j2a379b38ymsnn/LR-2018-BUG.swf?dl=0

Just Shoot Me
Legend
December 8, 2018

Wish this was posted earlier. I downloaded and open that catalog and then my system made 180MBs new catalogs.

Luckily I have a drive image created on 11/20 that I used to restore my system.

NitinGupta7
Adobe Employee
Adobe Employee
December 5, 2018

Pushp, could you please explain your workflow a little more, couldnt seem to understand it.

I am still getting separate values, for separate amount of pictures in the LR Catalog.

2k.Pandey
2k.PandeyAuthor
Known Participant
December 7, 2018

Hi Nitin!

So I'm exporting a one image catalog, and getting 180 MB lrcat file.  Not just that, even if I export just 1 file in the new catalog, its taking 10 times as long to export it as compared to the earlier version.


The only new thing I'm noticing is this new setting which I haven't seen earlier.


Going even further with my testing, I went to the menu and created a new catalog. Didn't import anything into it, and closed the catalog. Instantly upon closing it became a 180 MB file again. That catalog has just been created and it has literally nothing in it. Not a single imported file. Its still 180 MB file. Something is being written to the LRCAT file which is totally bloating its size.

Going totally crazy.

Just Shoot Me
Legend
December 7, 2018

I just created a new catalog and it was 1.5 MBs with no images imported.

 

 

Then I imported 18 images and it got slightly larger but not by much. 1.93 MBs

 

So I have no idea why your new Test catalog was 180 MBs in size. Are you sure it wasn't 1.80​ MBs?

JohanElzenga
Community Expert
Community Expert
December 4, 2018

The Lightroom catalog is a database containing metadata and edits. That means that the amount of metadata and the number of edits has a large impact on the catalog file size. So does the edit history. And that means that a catalog with fewer images (but lots of edits) can be bigger than a catalog with more images with fewer edits.

Adobe changed the catalog structure in Lightroom Classic. As a result, the catalog file is quite a bit smaller than it was in Lightroom 6 (mine shrunk by about 40% when I upgraded). It also means that it doesn’t compress that well anymore, because it is already compressed.

-- Johan W. Elzenga
2k.Pandey
2k.PandeyAuthor
Known Participant
December 4, 2018

Thanks for taking out the time to write all of that.. Being an Adobe certified expert in Photoshop, I have some idea about all of that.

Your answer doesn't answer my concern as if there's a particular catalog in question, which exports a sub catalog, containing only 5% of the images, its still similar size. No edits of images changed in between  Also, I'm only upgrading from the previous version. Not from the archaic 6.0. I have been on Classic CC for as long as it exists.

I have catalog backups (from recent version of LR before 8.0) from a few weeks ago which have 81% compression. These ones have 5-10% compression. Everything is comparable between these catalogs.

Legend
December 4, 2018

The catalog is a database, and as such, it probably has a minimum size, even if it had zero images. This may be what you are seeing.