Copy link to clipboard
Copied
Recently, I had been stitching a lot of 16MP photos into HDR and the resulting HDR photos into panoramas (by using LR Classic Merge in the background - ctrl-shift-H and M, since Adobe Camera Raw doesn't allow background merges, even though without any BATCH processing option, this is still a major PITA).
Since there was a lot of the merges queued, my start-up disk almost completely filled up even though I had almost 30GB free on the SSD. I have dedicated SSDs in RAID just for scratch files and working with video, these are much faster than the start-up SSD (256GB) and had ample space. Is there any way to tell LR to use only those for scratch files? Mind you, these were not previews getting generated, the space immediately got reclaimed when I quit LR, these were just temp files or maybe system memory (got 32GB which is not so big nowadays I admit) getting swapped to disk.
Still, inside Photoshop I can at least assign scratch disk to specific drives, which does work when merging big panoramas manually (about the most memory intensive operation I do there), although that does not seem to work for Camera Raw (which seems to rely only on system memory and results in swapping to disk in some operations).
So, is there any way to tell LR to use specific drives for any temporary files?
Copy link to clipboard
Copied
Oh, alright, thanks. That’s a valid point, and I do usually try to keep the free space bigger than that for TRIM and SSD internal housekeeping. That’s why I was even more concerned about the behaviour, since I had set all programs to use the other scratch drive.
Copy link to clipboard
Copied
Is it still unsolved? Did you find a way around it? I got pretty much the same scenario (only I have 37GB free from a 120GB OS disk) and a huge NAS, that I would love to use for a HDR panorama merge. I was watching the system to fill up without any of the temp directories to grow in size (at least not significantly). Btw FYI: on Mac the location of that temp dir in /var/folders is contained in the $TMPDIR variable.
Copy link to clipboard
Copied
Hi, unfortunately, I did not. Although as I wrote earlier, it's been quite some time since I needed to make big HDR+PANO merges, so I do not notice it now in my normal work. But from your and earlier replies, it seems it still might the same issue. You are right about the location, I did mention TMPDIR, although if I rememeber correctly, it was nested further in $TMPDIR/Cleanup At Startup/[numbered folder]/cr_sdk_[some numbers].tmp (Cleanup At Startup is AFAIK an obsolete folder name used for scratch back in PowerPC Macs OS 9, and the numbers make it somewhat difficult to link the file somewhere else, and I am wary of linking inside system folders).
I am sorry I can't help you with testing it again on my machine right now, I can try to do a retest with some really big HDR+PANO merges later.
Although I can give you one warning - even if we did find a workaround or Adobe fixed the issue, you should never use a NAS for any kind of intensive file operations like scratch files - the slow network access will slow it down to a crawl. You could probably get by with an USB3/Thunderbolt attached external drive in a pinch, if you are on a laptop. I use fast (separate from system drive) internal SSD for big scratch files (although the problem was that Adobe couldn't use it even if I tried) on desktop.
Copy link to clipboard
Copied
Frank, you can go about this any way you like, but it all boils down to your 256GB system drive. That's a problem waiting to happen, if it hasn't already. It's just too small.
Lightroom doesn't need a scratch disk as long as you have enough space for the OS to do its normal paging.
The real problem is the way the user account accumulates and fills up on both Mac and Windows. Nothing is ever removed here unless you clean out yourself. An application uninstall just leaves everything behind, so it keeps accumulating without limit. Bits and pieces add up.
On Windows there's an excellent utility called WinDirStat that gives you a graphic presentation of what fills up the drive and exactly where it is. I'm sure there's something similar for Mac.
Copy link to clipboard
Copied
D_Fosse I don't think you understand the issue, your comment isn't helpful and is condescending. There are no issues with user account files filling up, I normally have 150gb or so free on my system drive, it's only during a panorama merge that Lightroom uses 100gb of disc space. A 256gb system drive is fine for a Mac if mostly using external storage for everything but application installs. My Lightroom catalog and cache is on a separate ssd so the normal Lightroom fuctions aren't effected by the system drive size. It's only panorama merges that use system drive space. Closing Lightroom clears the cache and frees the space right back up, but this should be done at the end of the merge not only when closing Lightroom. Also this cache should be created on the selected camera raw cache or catalog location, and not on the system disc where Lightroom is installed. I have lots of fast SSD external storage however Lightroom will not use this for panorama merges which is a problem.
Copy link to clipboard
Copied
I'm just trying to explain how things actually work. Make a request to change it here: https://feedback.photoshop.com/photoshop_family/categories/photoshop_family_photoshop_lightroom
The system drive is usually the fastest drive available, especially nowadays when NVMe is the norm. So under most normal circumstances there are very good reasons to keep this on the system drive. But yes, for users with small system drives, where there simply isn't enough space, there should be an option to move it. I agree with that.
The point about the user account is valid, though, irrespective of whether you are on MacOS or Windows. It's not just Lightroom. Everything you do deposits various files in the user account, and most of it is never removed again unless you do it yourself. Things accumulate, that's the point.
Copy link to clipboard
Copied
Just for anyone still reading this thread and seeing D.Fosse's answer, as well as other answers about the small OS drive on Macs usually called Macintosh HD... I'm now having the same issue, and there is no time or money to spend on upgrading the internal drive of this Mac. The answer of 'get another larger hard drive' could be technically correct, but shows very little compassion. True, compassion doesn't translate well in written messages but something like 'unfortunately our product is hard wired to drop temporary files on the main disk, so there's nothing we can do but take this down and suggest to the developers that Lightroom Classic switches the code to use a scratch disk instead.' Well at least we'd have closure and a better feeling about using / paying for the product.
Maybe coding a scratch disk in place of the 'use $TMPDIR' code is tedious and staff doesn't exist that knows how to do it. It also may not be considered a good idea since concerns of Lightroom's performance slowing down may cause the developers to not want to change the code.
Either way, Lightroom panoramas with larger image sizes (such as what I have from my Sony A7R4) are basically screwed when you run low on your main internal drive's space.
Now we have to figure out how to make our machines boot off of a personal SAN or something. Maybe that's the future of home computing, where we can boot off of a virtual disk and then grow it as we need it.
Copy link to clipboard
Copied
Same problem here.
Thanks for your very detailed posts FrankWPhoto.
When doing big pano, my 32GB of RAM is not enough and Lightroom start using my main drive as a backup for storing temp files in order to be able to achieve the pano.
My main drive is an old SDD connected by a Sata cable. I was hoping to find some solution here to tell lightroom to use my NVME drive (where lightroom is, the photos, the catalog) but it seems that's not possible at the moment.
So a NVME PCIE 4.0 for OS drive seems to be a good investment when doing a lot of pano (whatching out for overheating).
Copy link to clipboard
Copied
This being a old posting, and one that appears to perhaps not get absolutely resolved (think some misunderstanding going on)
I recommend that you create your own new posting for your problem/
When you do, include your issue, your workflow, and your system info as Lightroom reports it (first line thru and including plug-in info, /Help/System Info/Copy/)
However, some comments
These for WIN OS
1. The Camera RAW CACHE used by Lightroom Classic is not the same as the OS Paging File, or TEMP directory
2. The Location and size (limitation) for the Camera RAW CACHE used by Lightroom Classic is set in /Preferences/Performance/ (discussed somewhere above)
3. If at all possible do not place the Camera RAW CACHE on the same hard drive as the OS Paging File (Temp directory of no importance). If the Camera RAW CACHE and the Paging file are on the same hard drive, they will compete for read/wrights on that hard drive.
4. The Camera RAW CACHE should be, if available, on a very fast Hard drive.
5. The Camera RAW CACHE can be on any physical hard drive (but no, not a SD card, etc)
6. Your Catalog can also take advantage of hard drive speed, and can be on any physical hard drive (not a SD card)
7. Your photos can be on any hard drive, but their is no advantage as to hard drive speed (and as you can assume, not a SD card, after import)
OH, and remember, photos are not stored in the Lightroom Catalog, and when you back up the catalog, you are not backing up photos.
Copy link to clipboard
Copied
It's not just panoramas that cause this issue, either! When you publish a large batch of files (I use jf Folder Publisher to publish all my edited RAW photos into a set of jpgs for normal work use, slideshows etc). The publish process gradulally fills up the SSD system drive (using up 100gb of space that was free beforehand (i.e. 20% of the disk was free before Lightrooom started exporting). The publish starts off lightning fast and gradually slows as the drive fills up. Eventauly its so slow that I stop the task, shut Lightroom (at which point the SSD free space miraculously reappears) and then reopen Lightroom to start the publishing again where I left off. My workaround is to split the folders into smaller publish batches, but that really shouldnt be necessary if Lightroom let you set an overspill drive for temporary storage during large batch operations. It's odd that Photoshop allows this even though it rarely needs to handle large batch operations, whereas Lightroom is designed for handling large numbers of files.
Copy link to clipboard
Copied
Happens to be when I am exporting my edited RAW files to jpg. Very frustrating! I even keep my catalog and Camera Raw directory on a brand new, empty Samsung 2TB drive. (I am running a MBP 2018)
Copy link to clipboard
Copied
What specific folder(s) are involved?
Copy link to clipboard
Copied
cr_sdk_
And no, this has not been solved.
To me, it happened generating 1:1 previews for a (very, very big) folder
Now my C: drive is stuffed by a 130GB temp file which is NOT TEMPORARY AT ALL.
Not restarting LRC or Windows deleted it. I will delete this by hand by it seems ridicolous to me.
Copy link to clipboard
Copied
are you sure that's not the Preview file that Lr generates?
Copy link to clipboard
Copied
Yes
Copy link to clipboard
Copied
Can confirm, this is not resolved in the latest versions. When trying to HDR merge many large MP files you will stumble across this. LRC appears to write the temp files it uses to the default user profile temp location, and not the user defined scratch space, thus you run the risk of filling up your C drive. WHen LR is closed you'll see it stay open for a while "cleaning up" and eventually this file is deleted. So it's something you likely never notice until it's a problem.
This is 100% something that needs to be changed by Adobe and something many more photographers will run into as MP counts are increasing, leading to more storage demands. For example, as it stands I could not make a 360 degree pano with my a7r4 without upgrading to a larger disk for my C drive (currently 180GB free). This is directly due to Adobe's programming and lack of customization for this cache file location.
Adobe, all scratch/cache should go to user defined areas. There should be no scratch files made outside what we define.
Copy link to clipboard
Copied
Yes, i've been trying to get this fixed in Ps for the past decade too. For whatever unknown reason, Smart Objects in Ps cache to the startup instead of the scratch.