Copy link to clipboard
Copied
Hi Everyone!
Experiencing two weird issues with respect to LRCAT file size and LRCAT backup size.
1st problem is Catalog size is more or less independent of number of photos. So 152 shot catalog and 25 shot catalog, are both 200 MBs unlike how it previously was. Here, I'm talking about the file size of the LRCAT file itself and nothing else. Just the catalog. As sometimes I've to export a 3 shot LRCAT file for the client when they already have the original RAW files and all I need to send them is the LRCAT file and nothing else.
2nd, compression ratio for LRCAT file (when compressed and saved by Lightroom) was around 80%. This was more or less similar to what you would achieve if you manually compressed an LRCAT file. While now theres barely 5-8% compression. Basically, 200 MB LRCAT would now be backed up as 180 MB compressed zip file. Earlier that was 10 MB or so. So all LRCAT backup folders are growing at a ginormous rate.
I hope that this is something that adobe can resolve and not something permanent with LR 8.0. As its making image transfer really difficult for people where good internet connection is still a luxury.
Is someone else experiencing this? Can someone from Adobe advise on when (or if) this will be fixed?
Thanks in advance for any help
In order to clean catalogs presenting the issue, I've found the following workaround.
1. Make sure the folder
Application Support/Adobe/CameraRaw/Tables
is empty.
2. Within an SQLite editor, delete all the rows in the table
Adobe_libraryImageDevelop3DLUTColorTable
for each plagued catalog (delete all the rows of records, not the table itself).
3.
As size of .lrcat files is not reduced yet, open in LR each catalog, optimize it then close it.
In all my non-infected catalogs the table Adobe_libraryImageDeve
...Copy link to clipboard
Copied
The Lightroom catalog is a database containing metadata and edits. That means that the amount of metadata and the number of edits has a large impact on the catalog file size. So does the edit history. And that means that a catalog with fewer images (but lots of edits) can be bigger than a catalog with more images with fewer edits.
Adobe changed the catalog structure in Lightroom Classic. As a result, the catalog file is quite a bit smaller than it was in Lightroom 6 (mine shrunk by about 40% when I upgraded). It also means that it doesn’t compress that well anymore, because it is already compressed.
Copy link to clipboard
Copied
Thanks for taking out the time to write all of that.. Being an Adobe certified expert in Photoshop, I have some idea about all of that.
Your answer doesn't answer my concern as if there's a particular catalog in question, which exports a sub catalog, containing only 5% of the images, its still similar size. No edits of images changed in between Also, I'm only upgrading from the previous version. Not from the archaic 6.0. I have been on Classic CC for as long as it exists.
I have catalog backups (from recent version of LR before 8.0) from a few weeks ago which have 81% compression. These ones have 5-10% compression. Everything is comparable between these catalogs.
Copy link to clipboard
Copied
The catalog is a database, and as such, it probably has a minimum size, even if it had zero images. This may be what you are seeing.
Copy link to clipboard
Copied
dj_paige wrote
The catalog is a database, and as such, it probably has a minimum size, even if it had zero images. This may be what you are seeing.
I understand that is true, but here's the thing. From an unaffected computer, a newly created LRCAT with Zero images has a size of around 2 MBs. While a similar catalog from the affected computer (or lightroom) has the LRCAT file with 180 MB size. You may want to check the file I've uploaded in a message above, in case its difficult for you to believe that. Its quite unbelievable for me as well.
Copy link to clipboard
Copied
I am also experiencing the same problem. All my catalogs are in between 200-250MB and it never happened like this before.
Copy link to clipboard
Copied
Pushp, could you please explain your workflow a little more, couldnt seem to understand it.
I am still getting separate values, for separate amount of pictures in the LR Catalog.
Copy link to clipboard
Copied
Hi Nitin!
So I'm exporting a one image catalog, and getting 180 MB lrcat file. Not just that, even if I export just 1 file in the new catalog, its taking 10 times as long to export it as compared to the earlier version.
The only new thing I'm noticing is this new setting which I haven't seen earlier.
Going even further with my testing, I went to the menu and created a new catalog. Didn't import anything into it, and closed the catalog. Instantly upon closing it became a 180 MB file again. That catalog has just been created and it has literally nothing in it. Not a single imported file. Its still 180 MB file. Something is being written to the LRCAT file which is totally bloating its size.
Going totally crazy.
Copy link to clipboard
Copied
I just created a new catalog and it was 1.5 MBs with no images imported.
Then I imported 18 images and it got slightly larger but not by much. 1.93 MBs
So I have no idea why your new Test catalog was 180 MBs in size. Are you sure it wasn't 1.80​ MBs?
Copy link to clipboard
Copied
Thanks for trying that out on your computer. I just recently found one one of my computers is not affected as well. And I can replicate what you just did there. But now what I'm going to do is upload the empty catalog for you to see (from the affected computer) and open for yourself. You'll see its totally empty. And of course its not 1.8 MBs. lol Its 180 Mega Bytes, which is roughly 0.18 GBs.
Dropbox - new test catalog.zip
As for your other question, I don't really know what are the 3D LUTs which are seen in the screenshot but other people have reported seeing that same thing without having 180 MB catalogs. I have also done a re-install without results. Issue remains.
Copy link to clipboard
Copied
Do you have the preference option to 'store presets with this catalog' turned on?
It was probably turned off earlier or you just installed new presets that increased the size of the stored presets within the catalog.
Copy link to clipboard
Copied
bhousto90 wrote
Do you have the preference option to 'store presets with this catalog' turned on?
It was probably turned off earlier or you just installed new presets that increased the size of the stored presets within the catalog.
Presets can be stored with the catalog. They are not stored inside the catalog file.
Copy link to clipboard
Copied
bhousto90 wrote
Do you have the preference option to 'store presets with this catalog' turned on?
It was probably turned off earlier or you just installed new presets that increased the size of the stored presets within the catalog.
bhousto90​ I don't have this turned on. I have checked this, and most other menu options to find anything that helps. I haven't. If you check the file I've shared above, you can see the catalog for yourself. There's nothing except the LRCAT file in it. Still its bloated. Plus, if you zip it, you won't find the usual compression. Someone from the tech team of Adobe would need to have a look at it to check what does the file contain which is causing is inordinate increase in its size.
Copy link to clipboard
Copied
What is the 3D LUT that is being copied to the cat export?
Copy link to clipboard
Copied
I just checked on my MacBook Pro. I created a new catalog for an online workshop I'm writing, and imported 87 photos into it. The catalog file is 3.7 MB...
Copy link to clipboard
Copied
This is also happening to my catalogs. I have not found any resolution. Did this issue resolve itself for you?
Copy link to clipboard
Copied
2k.Pandey wrote
Hi Everyone!
Experiencing two weird issues with respect to LRCAT file size and LRCAT backup size.
1st problem is Catalog size is more or less independent of number of photos. So 152 shot catalog and 25 shot catalog, are both 200 MBs unlike how it previously was. Here, I'm talking about the file size of the LRCAT file itself and nothing else. Just the catalog. As sometimes I've to export a 3 shot LRCAT file for the client when they already have the original RAW files and all I need to send them is the LRCAT file and nothing else.
2nd, compression ratio for LRCAT file (when compressed and saved by Lightroom) was around 80%. This was more or less similar to what you would achieve if you manually compressed an LRCAT file. While now theres barely 5-8% compression. Basically, 200 MB LRCAT would now be backed up as 180 MB compressed zip file. Earlier that was 10 MB or so. So all LRCAT backup folders are growing at a ginormous rate.
I hope that this is something that adobe can resolve and not something permanent with LR 8.0. As its making image transfer really difficult for people where good internet connection is still a luxury.
Is someone else experiencing this? Can someone from Adobe advise on when (or if) this will be fixed?
Thanks in advance for any help
Thanks 2k.Pandey for getting this over a call with me and experimenting with the issue.
The issue seems like a bug, but it spreads to every LR which opens the infected file. I was able to record the video while the issue was being created in a .SWF file. attached!
https://www.dropbox.com/s/7j2a379b38ymsnn/LR-2018-BUG.swf?dl=0
Copy link to clipboard
Copied
Wish this was posted earlier. I downloaded and open that catalog and then my system made 180MBs new catalogs.
Luckily I have a drive image created on 11/20 that I used to restore my system.
Copy link to clipboard
Copied
Checked the catalog, seeing that the 3D color LUT took up the majority of the size of the new catalog. The catalog stores a total of 1021 entries of custom 3D LUT (not from Adobe), presumably coming from some custom camera profiles/presets.
Lightroom caches unique 3D LUT both in the catalog and on the disk to facilitate transportation of catalogs among different machines and quick access to them at develop time. Lightroom does not keep a copy of the 3D LUT at each history step.
The catalog size increase for a newly created/exported catalog is a manifestation of this. It was introduced as part of the Lightroom Classic 7.3.
Given that a reference to a 3D LUT could be buried in one of a history step (customer can go back in the history step to reference a 3D LUT at any time). Lightroom catalog does not currently have an infrastructure to do proper reference counting of custom 3D LUT usage. So it just includes a copy of every custom 3D LUT ever used.
We have to consider this case of multiple catalog usage scenario and see if we can optimize it. Thanks for the report.
P.S. The 3D LUT is also cached locally on disk at "~/Library/Application Support/Adobe/CameraRaw/Tables" on macOS. There is a similar Window directory location.
Copy link to clipboard
Copied
Does this mean potentially rolling back to 7.2 would alleviate this issues until there is a rollout fix in the next update? Its causing me all types of time delays each day in waiting for uploads/downloads that are 20 times as big as before!
Copy link to clipboard
Copied
Thank you for your reply.
simonsaith wrote
Checked the catalog, seeing that the 3D color LUT took up the majority of the size of the new catalog. The catalog stores a total of 1021 entries of custom 3D LUT (not from Adobe), presumably coming from some custom camera profiles/presets.
Lightroom caches unique 3D LUT both in the catalog and on the disk to facilitate transportation of catalogs among different machines and quick access to them at develop time. Lightroom does not keep a copy of the 3D LUT at each history step.
The catalog size increase for a newly created/exported catalog is a manifestation of this. It was introduced as part of the Lightroom Classic 7.3.
Given that a reference to a 3D LUT could be buried in one of a history step (customer can go back in the history step to reference a 3D LUT at any time). Lightroom catalog does not currently have an infrastructure to do proper reference counting of custom 3D LUT usage. So it just includes a copy of every custom 3D LUT ever used.
We have to consider this case of multiple catalog usage scenario and see if we can optimize it. Thanks for the report.
P.S. The 3D LUT is also cached locally on disk at "~/Library/Application Support/Adobe/CameraRaw/Tables" on macOS. There is a similar Window directory location.
I checked the directory you mentioned (TABLES) and I found 1021 files I think which totalled up to 172 MBs which is also close to the resulting catalog size. So its definitely relating to that. Also, whats interesting is, even after deletion of these files, they're automatically created again in a second after any new catalog is created or opened on an already infected machine. I checked a non-infected machine and this directory for them, is totally empty. Deleting camera profiles hasn't worked either.
I am trying various permutations and combinations to alleviate the issue as two of my main machines are infected and work on them isn't possible because of this. I usually export a lot of sub-catalogs to send to my editors, and they're all super heavy even though they have very less images in them.
Copy link to clipboard
Copied
In order to clean catalogs presenting the issue, I've found the following workaround.
1. Make sure the folder
Application Support/Adobe/CameraRaw/Tables
is empty.
2. Within an SQLite editor, delete all the rows in the table
Adobe_libraryImageDevelop3DLUTColorTable
for each plagued catalog (delete all the rows of records, not the table itself).
3.
As size of .lrcat files is not reduced yet, open in LR each catalog, optimize it then close it.
In all my non-infected catalogs the table Adobe_libraryImageDevelop3DLUTColorTable is empty, therefore emptying this table outside LR should not produce any damage to the catalog itself.
Copy link to clipboard
Copied
@path72 Thank you for your response. Your method does fix the problem and I would have liked it more if Adobe addressed this issue coz its plauging catalogs even now (9.4). There's a client of mine who has so many infected catalogs, that they've stopped caring. So much bandwidth and storage space is being wasted.
I'll communicate this solution to them and hopefully it can be resolved.
Copy link to clipboard
Copied
I'm having this same problem in V10.2, but the "Tables" file/folder doesn't exist. There is a CameraRaw folder and a Camera Raw 6. Searched "Tables" but haven't found anything. Uninstalled and re installed with no success. Any help would be greatly appreciated.
Copy link to clipboard
Copied