Lightroom Classic CC 8.0 large sized backups and LRCAT file

Community-Einsteiger ,
Dec 04, 2018

Link in Zwischenablage kopieren

Kopiert

Hi Everyone!

Experiencing two weird issues with respect to LRCAT file size and LRCAT backup size.

1st problem is Catalog size is more or less independent of number of photos.  So 152 shot catalog and 25 shot catalog, are both 200 MBs unlike how it previously was. Here, I'm talking about the file size of the LRCAT file itself and nothing else. Just the catalog. As sometimes I've to export a 3 shot LRCAT file for the client when they already have the original RAW files and all I need to send them is the LRCAT file and nothing else.

2nd, compression ratio for LRCAT file (when compressed and saved by Lightroom) was around 80%. This was more or less similar to what you would achieve if you manually compressed an LRCAT file. While now theres barely 5-8% compression. Basically, 200 MB LRCAT would now be backed up as 180 MB compressed zip file. Earlier that was 10 MB or so. So all LRCAT backup folders are growing at a ginormous rate.

I hope that this is something that adobe can resolve and not something permanent with LR 8.0. As its making image transfer really difficult for people where good internet connection is still a luxury.

Is someone else experiencing this? Can someone from Adobe advise on when (or if) this will be fixed?

Thanks in advance for any help

Zugriffe

1.6K

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen

1 Correct Answer

Neu hier , Feb 12, 2019
path72 Neu hier , Feb 12, 2019
In order to clean catalogs presenting the issue, I've found the following workaround.1. Make sure the folderApplication Support/Adobe/CameraRaw/Tablesis empty.2. Within an SQLite editor, delete all the rows in the tableAdobe_libraryImageDevelop3DLUTColorTablefor each plagued catalog (delete all the rows of records, not the table itself).3.As size of .lrcat files is not reduced yet, open in LR each catalog, optimize it then close it.In all my non-infected catalogs the table Adobe_libraryImageDeve...

Likes

Übersetzen

Übersetzen
Jump to answer Zur Antwort springen
Adobe Community-Experte ,
Dec 04, 2018

Link in Zwischenablage kopieren

Kopiert

The Lightroom catalog is a database containing metadata and edits. That means that the amount of metadata and the number of edits has a large impact on the catalog file size. So does the edit history. And that means that a catalog with fewer images (but lots of edits) can be bigger than a catalog with more images with fewer edits.

Adobe changed the catalog structure in Lightroom Classic. As a result, the catalog file is quite a bit smaller than it was in Lightroom 6 (mine shrunk by about 40% when I upgraded). It also means that it doesn’t compress that well anymore, because it is already compressed.

-- Johan W. Elzenga

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Community-Einsteiger ,
Dec 04, 2018

Link in Zwischenablage kopieren

Kopiert

Thanks for taking out the time to write all of that.. Being an Adobe certified expert in Photoshop, I have some idea about all of that.

Your answer doesn't answer my concern as if there's a particular catalog in question, which exports a sub catalog, containing only 5% of the images, its still similar size. No edits of images changed in between  Also, I'm only upgrading from the previous version. Not from the archaic 6.0. I have been on Classic CC for as long as it exists.

I have catalog backups (from recent version of LR before 8.0) from a few weeks ago which have 81% compression. These ones have 5-10% compression. Everything is comparable between these catalogs.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
LEGENDE ,
Dec 04, 2018

Link in Zwischenablage kopieren

Kopiert

The catalog is a database, and as such, it probably has a minimum size, even if it had zero images. This may be what you are seeing.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Community-Einsteiger ,
Dec 07, 2018

Link in Zwischenablage kopieren

Kopiert

dj_paige  wrote

The catalog is a database, and as such, it probably has a minimum size, even if it had zero images. This may be what you are seeing.


I understand that is true, but here's the thing. From an unaffected computer, a newly created LRCAT with Zero images has a size of around 2 MBs. While a similar catalog from the affected computer (or lightroom) has the LRCAT file with 180 MB size. You may want to check the file I've uploaded in a message above, in case its difficult for you to believe that. Its quite unbelievable for me as well.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Neu hier ,
Dec 04, 2018

Link in Zwischenablage kopieren

Kopiert

I am also experiencing the same problem. All my catalogs are in between 200-250MB and it never happened like this before.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Adobe-Mitarbeiter ,
Dec 05, 2018

Link in Zwischenablage kopieren

Kopiert

Pushp, could you please explain your workflow a little more, couldnt seem to understand it.

I am still getting separate values, for separate amount of pictures in the LR Catalog.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Community-Einsteiger ,
Dec 07, 2018

Link in Zwischenablage kopieren

Kopiert

Hi Nitin!

So I'm exporting a one image catalog, and getting 180 MB lrcat file.  Not just that, even if I export just 1 file in the new catalog, its taking 10 times as long to export it as compared to the earlier version.


The only new thing I'm noticing is this new setting which I haven't seen earlier.


Screenshot (70).png

Going even further with my testing, I went to the menu and created a new catalog. Didn't import anything into it, and closed the catalog. Instantly upon closing it became a 180 MB file again. That catalog has just been created and it has literally nothing in it. Not a single imported file. Its still 180 MB file. Something is being written to the LRCAT file which is totally bloating its size.

Going totally crazy.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
LEGENDE ,
Dec 07, 2018

Link in Zwischenablage kopieren

Kopiert

I just created a new catalog and it was 1.5 MBs with no images imported.

 

 

Then I imported 18 images and it got slightly larger but not by much. 1.93 MBs

 

So I have no idea why your new Test catalog was 180 MBs in size. Are you sure it wasn't 1.80​ MBs?

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Community-Einsteiger ,
Dec 07, 2018

Link in Zwischenablage kopieren

Kopiert

Thanks for trying that out on your computer. I just recently found one one of my computers is not affected as well. And I can replicate what you just did there. But now what I'm going to do is upload the empty catalog for you to see (from the affected computer) and open for yourself. You'll see its totally empty. And of course its not 1.8 MBs. lol Its 180 Mega Bytes, which is roughly 0.18 GBs.

Dropbox - new test catalog.zip

As for your other question, I don't really know what are the 3D LUTs which are seen in the screenshot but other people have reported seeing that same thing without having 180 MB catalogs. I have also done a re-install without results. Issue remains.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Enthusiast ,
Dec 07, 2018

Link in Zwischenablage kopieren

Kopiert

Do you have the preference option to 'store presets with this catalog' turned on?

It was probably turned off earlier or you just installed new presets that increased the size of the stored presets within the catalog.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Adobe Community-Experte ,
Dec 07, 2018

Link in Zwischenablage kopieren

Kopiert

bhousto90  wrote

Do you have the preference option to 'store presets with this catalog' turned on?

It was probably turned off earlier or you just installed new presets that increased the size of the stored presets within the catalog.

Presets can be stored with the catalog. They are not stored inside the catalog file.

-- Johan W. Elzenga

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Community-Einsteiger ,
Dec 07, 2018

Link in Zwischenablage kopieren

Kopiert

bhousto90  wrote

Do you have the preference option to 'store presets with this catalog' turned on?

It was probably turned off earlier or you just installed new presets that increased the size of the stored presets within the catalog.

bhousto90​ I don't have this turned on. I have checked this, and most other menu options to find anything that helps. I haven't. If you check the file I've shared above, you can see the catalog for yourself. There's nothing except the LRCAT file in it. Still its bloated. Plus, if you zip it, you won't find the usual compression. Someone from the tech team of Adobe would need to have a look at it to check what does the file contain which is causing is inordinate increase in its size.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
LEGENDE ,
Dec 07, 2018

Link in Zwischenablage kopieren

Kopiert

What is the 3D LUT that is being copied to the cat export?

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Adobe Community-Experte ,
Dec 07, 2018

Link in Zwischenablage kopieren

Kopiert

I just checked on my MacBook Pro. I created a new catalog for an online workshop I'm writing, and imported 87 photos into it. The catalog file is 3.7 MB...

-- Johan W. Elzenga

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Neu hier ,
Aug 19, 2019

Link in Zwischenablage kopieren

Kopiert

This is also happening to my catalogs.  I have not found any resolution.  Did this issue resolve itself for you?

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Adobe-Mitarbeiter ,
Dec 08, 2018

Link in Zwischenablage kopieren

Kopiert

2k.Pandey  wrote

Hi Everyone!

Experiencing two weird issues with respect to LRCAT file size and LRCAT backup size.

1st problem is Catalog size is more or less independent of number of photos.  So 152 shot catalog and 25 shot catalog, are both 200 MBs unlike how it previously was. Here, I'm talking about the file size of the LRCAT file itself and nothing else. Just the catalog. As sometimes I've to export a 3 shot LRCAT file for the client when they already have the original RAW files and all I need to send them is the LRCAT file and nothing else.

2nd, compression ratio for LRCAT file (when compressed and saved by Lightroom) was around 80%. This was more or less similar to what you would achieve if you manually compressed an LRCAT file. While now theres barely 5-8% compression. Basically, 200 MB LRCAT would now be backed up as 180 MB compressed zip file. Earlier that was 10 MB or so. So all LRCAT backup folders are growing at a ginormous rate.

I hope that this is something that adobe can resolve and not something permanent with LR 8.0. As its making image transfer really difficult for people where good internet connection is still a luxury.

Is someone else experiencing this? Can someone from Adobe advise on when (or if) this will be fixed?

Thanks in advance for any help

Thanks 2k.Pandey for getting this over a call with me and experimenting with the issue.

The issue seems like a bug, but it spreads to every LR which opens the infected file. I was able to record the video while the issue was being created in a .SWF file. attached!

https://www.dropbox.com/s/7j2a379b38ymsnn/LR-2018-BUG.swf?dl=0

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
LEGENDE ,
Dec 08, 2018

Link in Zwischenablage kopieren

Kopiert

Wish this was posted earlier. I downloaded and open that catalog and then my system made 180MBs new catalogs.

Luckily I have a drive image created on 11/20 that I used to restore my system.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Adobe-Mitarbeiter ,
Dec 10, 2018

Link in Zwischenablage kopieren

Kopiert

Checked the catalog, seeing that the 3D color LUT took up the majority of the size of the new catalog. The catalog stores a total of 1021 entries of custom 3D LUT (not from Adobe), presumably coming from some custom camera profiles/presets.

Lightroom caches unique 3D LUT both in the catalog and on the disk to facilitate transportation of catalogs among different machines and quick access to them at develop time. Lightroom does not keep a copy of the 3D LUT at each history step.

The catalog size increase for a newly created/exported catalog is a manifestation of this. It was introduced as part of the Lightroom Classic 7.3.

Given that a reference to a 3D LUT could be buried in one of a history step (customer can go back in the history step to reference a 3D LUT at any time). Lightroom catalog does not currently have an infrastructure to do proper reference counting of custom 3D LUT usage. So it just includes a copy of every custom 3D LUT ever used.

We have to consider this case of multiple catalog usage scenario and see if we can optimize it. Thanks for the report.

P.S. The 3D LUT is also cached locally on disk at "~/Library/Application Support/Adobe/CameraRaw/Tables" on macOS. There is a similar Window directory location.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Community-Einsteiger ,
Dec 10, 2018

Link in Zwischenablage kopieren

Kopiert

Does this mean potentially rolling back to 7.2 would alleviate this issues until there is a rollout fix in the next update? Its causing me all types of time delays each day in waiting for uploads/downloads that are 20 times as big as before!

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Community-Einsteiger ,
Dec 24, 2018

Link in Zwischenablage kopieren

Kopiert

Thank you for your reply.

simonsaith  wrote

Checked the catalog, seeing that the 3D color LUT took up the majority of the size of the new catalog. The catalog stores a total of 1021 entries of custom 3D LUT (not from Adobe), presumably coming from some custom camera profiles/presets.

Lightroom caches unique 3D LUT both in the catalog and on the disk to facilitate transportation of catalogs among different machines and quick access to them at develop time. Lightroom does not keep a copy of the 3D LUT at each history step.

The catalog size increase for a newly created/exported catalog is a manifestation of this. It was introduced as part of the Lightroom Classic 7.3.

Given that a reference to a 3D LUT could be buried in one of a history step (customer can go back in the history step to reference a 3D LUT at any time). Lightroom catalog does not currently have an infrastructure to do proper reference counting of custom 3D LUT usage. So it just includes a copy of every custom 3D LUT ever used.

We have to consider this case of multiple catalog usage scenario and see if we can optimize it. Thanks for the report.

P.S. The 3D LUT is also cached locally on disk at "~/Library/Application Support/Adobe/CameraRaw/Tables" on macOS. There is a similar Window directory location.

I checked the directory you mentioned (TABLES) and I found 1021 files I think which totalled up to 172 MBs which is also close to the resulting catalog size. So its definitely relating to that. Also, whats interesting is, even after deletion of these files, they're automatically created again in a second after any new catalog is created or opened on an already infected machine. I checked a non-infected machine and this directory for them, is totally empty. Deleting camera profiles hasn't worked either.

I am trying various permutations and combinations to alleviate the issue as two of my main machines are infected and work on them isn't possible because of this. I usually export a lot of sub-catalogs to send to my editors, and they're all super heavy even though they have very less images in them.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Neu hier ,
Feb 12, 2019

Link in Zwischenablage kopieren

Kopiert

In order to clean catalogs presenting the issue, I've found the following workaround.

1. Make sure the folder

Application Support/Adobe/CameraRaw/Tables

is empty.

2. Within an SQLite editor, delete all the rows in the table

Adobe_libraryImageDevelop3DLUTColorTable

for each plagued catalog (delete all the rows of records, not the table itself).

3.

As size of .lrcat files is not reduced yet, open in LR each catalog, optimize it then close it.

In all my non-infected catalogs the table Adobe_libraryImageDevelop3DLUTColorTable is empty, therefore emptying this table outside LR should not produce any damage to the catalog itself.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Community-Einsteiger ,
Oct 27, 2020

Link in Zwischenablage kopieren

Kopiert

@path72 Thank you for your response. Your method does fix the problem and I would have liked it more if Adobe addressed this issue coz its plauging catalogs even now (9.4). There's a client of mine who has so many infected catalogs, that they've stopped caring. So much bandwidth and storage space is being wasted. 

I'll communicate this solution to them and hopefully it can be resolved.

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Adobe Community-Experte ,
Dec 08, 2018

Link in Zwischenablage kopieren

Kopiert

Opened the catalog file linked above in a text editor and it appears that the 180MB is a gigantic color profile set as the default soft proof profile. It appears Lightroom stores this profile in the catalog file! Try changing the default proof profile to something simply like sRGB and then exporting a catalog.

Also if it hasn't been mentioned before submit a bug report to https://feedback.photoshop.com

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen
Community-Einsteiger ,
Dec 09, 2018

Link in Zwischenablage kopieren

Kopiert

Hey Mate,

I checked my proof profile and it was still set to Adobe 1998.. same result changing it to sRGB. What were you looking for inside the catalog file with the text editor? Wondering if I can try change it this way, been trying everything and cant stop the file size from exploding on closing any new catalog.

Thanks!

Likes

Übersetzen

Übersetzen

Melden

Melden
Antworten
Community-Richtlinien
Seien Sie freundlich und respektvoll, geben Sie die ursprüngliche Quelle der Inhalte an und suchen Sie vor dem Absenden Ihres Beitrags nach Duplikaten. Weitere Informationen