Lightroom Classic CC 8.0 large sized backups and LRCAT file

Explorer ,
Dec 04, 2018 Dec 04, 2018

Copy link to clipboard

Copied

Hi Everyone!

Experiencing two weird issues with respect to LRCAT file size and LRCAT backup size.

1st problem is Catalog size is more or less independent of number of photos.  So 152 shot catalog and 25 shot catalog, are both 200 MBs unlike how it previously was. Here, I'm talking about the file size of the LRCAT file itself and nothing else. Just the catalog. As sometimes I've to export a 3 shot LRCAT file for the client when they already have the original RAW files and all I need to send them is the LRCAT file and nothing else.

2nd, compression ratio for LRCAT file (when compressed and saved by Lightroom) was around 80%. This was more or less similar to what you would achieve if you manually compressed an LRCAT file. While now theres barely 5-8% compression. Basically, 200 MB LRCAT would now be backed up as 180 MB compressed zip file. Earlier that was 10 MB or so. So all LRCAT backup folders are growing at a ginormous rate.

I hope that this is something that adobe can resolve and not something permanent with LR 8.0. As its making image transfer really difficult for people where good internet connection is still a luxury.

Is someone else experiencing this? Can someone from Adobe advise on when (or if) this will be fixed?

Thanks in advance for any help

Views

1.9K

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines

correct answers 1 Correct Answer

New Here , Feb 12, 2019 Feb 12, 2019
In order to clean catalogs presenting the issue, I've found the following workaround.1. Make sure the folderApplication Support/Adobe/CameraRaw/Tablesis empty.2. Within an SQLite editor, delete all the rows in the tableAdobe_libraryImageDevelop3DLUTColorTablefor each plagued catalog (delete all the rows of records, not the table itself).3.As size of .lrcat files is not reduced yet, open in LR each catalog, optimize it then close it.In all my non-infected catalogs the table Adobe_libraryImageDeve...

Likes

Translate

Translate
Adobe Community Professional ,
Dec 04, 2018 Dec 04, 2018

Copy link to clipboard

Copied

The Lightroom catalog is a database containing metadata and edits. That means that the amount of metadata and the number of edits has a large impact on the catalog file size. So does the edit history. And that means that a catalog with fewer images (but lots of edits) can be bigger than a catalog with more images with fewer edits.

Adobe changed the catalog structure in Lightroom Classic. As a result, the catalog file is quite a bit smaller than it was in Lightroom 6 (mine shrunk by about 40% when I upgraded). It also means that it doesn’t compress that well anymore, because it is already compressed.

-- Johan W. Elzenga

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Dec 04, 2018 Dec 04, 2018

Copy link to clipboard

Copied

Thanks for taking out the time to write all of that.. Being an Adobe certified expert in Photoshop, I have some idea about all of that.

Your answer doesn't answer my concern as if there's a particular catalog in question, which exports a sub catalog, containing only 5% of the images, its still similar size. No edits of images changed in between  Also, I'm only upgrading from the previous version. Not from the archaic 6.0. I have been on Classic CC for as long as it exists.

I have catalog backups (from recent version of LR before 8.0) from a few weeks ago which have 81% compression. These ones have 5-10% compression. Everything is comparable between these catalogs.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Dec 04, 2018 Dec 04, 2018

Copy link to clipboard

Copied

The catalog is a database, and as such, it probably has a minimum size, even if it had zero images. This may be what you are seeing.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Dec 07, 2018 Dec 07, 2018

Copy link to clipboard

Copied

dj_paige  wrote

The catalog is a database, and as such, it probably has a minimum size, even if it had zero images. This may be what you are seeing.


I understand that is true, but here's the thing. From an unaffected computer, a newly created LRCAT with Zero images has a size of around 2 MBs. While a similar catalog from the affected computer (or lightroom) has the LRCAT file with 180 MB size. You may want to check the file I've uploaded in a message above, in case its difficult for you to believe that. Its quite unbelievable for me as well.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Dec 04, 2018 Dec 04, 2018

Copy link to clipboard

Copied

I am also experiencing the same problem. All my catalogs are in between 200-250MB and it never happened like this before.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Employee ,
Dec 05, 2018 Dec 05, 2018

Copy link to clipboard

Copied

Pushp, could you please explain your workflow a little more, couldnt seem to understand it.

I am still getting separate values, for separate amount of pictures in the LR Catalog.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Dec 07, 2018 Dec 07, 2018

Copy link to clipboard

Copied

Hi Nitin!

So I'm exporting a one image catalog, and getting 180 MB lrcat file.  Not just that, even if I export just 1 file in the new catalog, its taking 10 times as long to export it as compared to the earlier version.


The only new thing I'm noticing is this new setting which I haven't seen earlier.


Screenshot (70).png

Going even further with my testing, I went to the menu and created a new catalog. Didn't import anything into it, and closed the catalog. Instantly upon closing it became a 180 MB file again. That catalog has just been created and it has literally nothing in it. Not a single imported file. Its still 180 MB file. Something is being written to the LRCAT file which is totally bloating its size.

Going totally crazy.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Dec 07, 2018 Dec 07, 2018

Copy link to clipboard

Copied

I just created a new catalog and it was 1.5 MBs with no images imported.

 

 

Then I imported 18 images and it got slightly larger but not by much. 1.93 MBs

 

So I have no idea why your new Test catalog was 180 MBs in size. Are you sure it wasn't 1.80​ MBs?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Dec 07, 2018 Dec 07, 2018

Copy link to clipboard

Copied

Thanks for trying that out on your computer. I just recently found one one of my computers is not affected as well. And I can replicate what you just did there. But now what I'm going to do is upload the empty catalog for you to see (from the affected computer) and open for yourself. You'll see its totally empty. And of course its not 1.8 MBs. lol Its 180 Mega Bytes, which is roughly 0.18 GBs.

Dropbox - new test catalog.zip

As for your other question, I don't really know what are the 3D LUTs which are seen in the screenshot but other people have reported seeing that same thing without having 180 MB catalogs. I have also done a re-install without results. Issue remains.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Enthusiast ,
Dec 07, 2018 Dec 07, 2018

Copy link to clipboard

Copied

Do you have the preference option to 'store presets with this catalog' turned on?

It was probably turned off earlier or you just installed new presets that increased the size of the stored presets within the catalog.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Dec 07, 2018 Dec 07, 2018

Copy link to clipboard

Copied

bhousto90  wrote

Do you have the preference option to 'store presets with this catalog' turned on?

It was probably turned off earlier or you just installed new presets that increased the size of the stored presets within the catalog.

Presets can be stored with the catalog. They are not stored inside the catalog file.

-- Johan W. Elzenga

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Dec 07, 2018 Dec 07, 2018

Copy link to clipboard

Copied

bhousto90  wrote

Do you have the preference option to 'store presets with this catalog' turned on?

It was probably turned off earlier or you just installed new presets that increased the size of the stored presets within the catalog.

bhousto90​ I don't have this turned on. I have checked this, and most other menu options to find anything that helps. I haven't. If you check the file I've shared above, you can see the catalog for yourself. There's nothing except the LRCAT file in it. Still its bloated. Plus, if you zip it, you won't find the usual compression. Someone from the tech team of Adobe would need to have a look at it to check what does the file contain which is causing is inordinate increase in its size.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Dec 07, 2018 Dec 07, 2018

Copy link to clipboard

Copied

What is the 3D LUT that is being copied to the cat export?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Dec 07, 2018 Dec 07, 2018

Copy link to clipboard

Copied

I just checked on my MacBook Pro. I created a new catalog for an online workshop I'm writing, and imported 87 photos into it. The catalog file is 3.7 MB...

-- Johan W. Elzenga

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Aug 19, 2019 Aug 19, 2019

Copy link to clipboard

Copied

This is also happening to my catalogs.  I have not found any resolution.  Did this issue resolve itself for you?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Employee ,
Dec 08, 2018 Dec 08, 2018

Copy link to clipboard

Copied

2k.Pandey  wrote

Hi Everyone!

Experiencing two weird issues with respect to LRCAT file size and LRCAT backup size.

1st problem is Catalog size is more or less independent of number of photos.  So 152 shot catalog and 25 shot catalog, are both 200 MBs unlike how it previously was. Here, I'm talking about the file size of the LRCAT file itself and nothing else. Just the catalog. As sometimes I've to export a 3 shot LRCAT file for the client when they already have the original RAW files and all I need to send them is the LRCAT file and nothing else.

2nd, compression ratio for LRCAT file (when compressed and saved by Lightroom) was around 80%. This was more or less similar to what you would achieve if you manually compressed an LRCAT file. While now theres barely 5-8% compression. Basically, 200 MB LRCAT would now be backed up as 180 MB compressed zip file. Earlier that was 10 MB or so. So all LRCAT backup folders are growing at a ginormous rate.

I hope that this is something that adobe can resolve and not something permanent with LR 8.0. As its making image transfer really difficult for people where good internet connection is still a luxury.

Is someone else experiencing this? Can someone from Adobe advise on when (or if) this will be fixed?

Thanks in advance for any help

Thanks 2k.Pandey for getting this over a call with me and experimenting with the issue.

The issue seems like a bug, but it spreads to every LR which opens the infected file. I was able to record the video while the issue was being created in a .SWF file. attached!

https://www.dropbox.com/s/7j2a379b38ymsnn/LR-2018-BUG.swf?dl=0

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Dec 08, 2018 Dec 08, 2018

Copy link to clipboard

Copied

Wish this was posted earlier. I downloaded and open that catalog and then my system made 180MBs new catalogs.

Luckily I have a drive image created on 11/20 that I used to restore my system.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Employee ,
Dec 10, 2018 Dec 10, 2018

Copy link to clipboard

Copied

Checked the catalog, seeing that the 3D color LUT took up the majority of the size of the new catalog. The catalog stores a total of 1021 entries of custom 3D LUT (not from Adobe), presumably coming from some custom camera profiles/presets.

Lightroom caches unique 3D LUT both in the catalog and on the disk to facilitate transportation of catalogs among different machines and quick access to them at develop time. Lightroom does not keep a copy of the 3D LUT at each history step.

The catalog size increase for a newly created/exported catalog is a manifestation of this. It was introduced as part of the Lightroom Classic 7.3.

Given that a reference to a 3D LUT could be buried in one of a history step (customer can go back in the history step to reference a 3D LUT at any time). Lightroom catalog does not currently have an infrastructure to do proper reference counting of custom 3D LUT usage. So it just includes a copy of every custom 3D LUT ever used.

We have to consider this case of multiple catalog usage scenario and see if we can optimize it. Thanks for the report.

P.S. The 3D LUT is also cached locally on disk at "~/Library/Application Support/Adobe/CameraRaw/Tables" on macOS. There is a similar Window directory location.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Dec 10, 2018 Dec 10, 2018

Copy link to clipboard

Copied

Does this mean potentially rolling back to 7.2 would alleviate this issues until there is a rollout fix in the next update? Its causing me all types of time delays each day in waiting for uploads/downloads that are 20 times as big as before!

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Dec 24, 2018 Dec 24, 2018

Copy link to clipboard

Copied

Thank you for your reply.

simonsaith  wrote

Checked the catalog, seeing that the 3D color LUT took up the majority of the size of the new catalog. The catalog stores a total of 1021 entries of custom 3D LUT (not from Adobe), presumably coming from some custom camera profiles/presets.

Lightroom caches unique 3D LUT both in the catalog and on the disk to facilitate transportation of catalogs among different machines and quick access to them at develop time. Lightroom does not keep a copy of the 3D LUT at each history step.

The catalog size increase for a newly created/exported catalog is a manifestation of this. It was introduced as part of the Lightroom Classic 7.3.

Given that a reference to a 3D LUT could be buried in one of a history step (customer can go back in the history step to reference a 3D LUT at any time). Lightroom catalog does not currently have an infrastructure to do proper reference counting of custom 3D LUT usage. So it just includes a copy of every custom 3D LUT ever used.

We have to consider this case of multiple catalog usage scenario and see if we can optimize it. Thanks for the report.

P.S. The 3D LUT is also cached locally on disk at "~/Library/Application Support/Adobe/CameraRaw/Tables" on macOS. There is a similar Window directory location.

I checked the directory you mentioned (TABLES) and I found 1021 files I think which totalled up to 172 MBs which is also close to the resulting catalog size. So its definitely relating to that. Also, whats interesting is, even after deletion of these files, they're automatically created again in a second after any new catalog is created or opened on an already infected machine. I checked a non-infected machine and this directory for them, is totally empty. Deleting camera profiles hasn't worked either.

I am trying various permutations and combinations to alleviate the issue as two of my main machines are infected and work on them isn't possible because of this. I usually export a lot of sub-catalogs to send to my editors, and they're all super heavy even though they have very less images in them.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Feb 12, 2019 Feb 12, 2019

Copy link to clipboard

Copied

In order to clean catalogs presenting the issue, I've found the following workaround.

1. Make sure the folder

Application Support/Adobe/CameraRaw/Tables

is empty.

2. Within an SQLite editor, delete all the rows in the table

Adobe_libraryImageDevelop3DLUTColorTable

for each plagued catalog (delete all the rows of records, not the table itself).

3.

As size of .lrcat files is not reduced yet, open in LR each catalog, optimize it then close it.

In all my non-infected catalogs the table Adobe_libraryImageDevelop3DLUTColorTable is empty, therefore emptying this table outside LR should not produce any damage to the catalog itself.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Oct 27, 2020 Oct 27, 2020

Copy link to clipboard

Copied

@path72 Thank you for your response. Your method does fix the problem and I would have liked it more if Adobe addressed this issue coz its plauging catalogs even now (9.4). There's a client of mine who has so many infected catalogs, that they've stopped caring. So much bandwidth and storage space is being wasted. 

I'll communicate this solution to them and hopefully it can be resolved.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
May 29, 2021 May 29, 2021

Copy link to clipboard

Copied

I'm having this same problem in V10.2, but the "Tables" file/folder doesn't exist.  There is a CameraRaw folder and a Camera Raw 6.  Searched "Tables" but haven't found anything.  Uninstalled and re installed with no success.  Any help would be greatly appreciated.

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
May 29, 2021 May 29, 2021

Copy link to clipboard

Copied

LATEST
Are you on a windows or mac machine?
The path to the LUT folder has been noted in one of the other answers on
the thread. You may refer to that for the same.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines