Known Participant

Question

LR Classic - changing the temp (scratch) drive? It fills up my start-up drive completely with some operations

Forum|Forum|8 years ago
December 24, 2017
10 replies
24513 views

Recently, I had been stitching a lot of 16MP photos into HDR and the resulting HDR photos into panoramas (by using LR Classic Merge in the background - ctrl-shift-H and M, since Adobe Camera Raw doesn't allow background merges, even though without any BATCH processing option, this is still a major PITA).

Since there was a lot of the merges queued, my start-up disk almost completely filled up even though I had almost 30GB free on the SSD. I have dedicated SSDs in RAID just for scratch files and working with video, these are much faster than the start-up SSD (256GB) and had ample space. Is there any way to tell LR to use only those for scratch files? Mind you, these were not previews getting generated, the space immediately got reclaimed when I quit LR, these were just temp files or maybe system memory (got 32GB which is not so big nowadays I admit) getting swapped to disk.

Still, inside Photoshop I can at least assign scratch disk to specific drives, which does work when merging big panoramas manually (about the most memory intensive operation I do there), although that does not seem to work for Camera Raw (which seems to rely only on system memory and results in swapping to disk in some operations).

So, is there any way to tell LR to use specific drives for any temporary files?

This topic has been closed for replies.

Earth Oliver

Legend

Yes, i've been trying to get this fixed in Ps for the past decade too. For whatever unknown reason, Smart Objects in Ps cache to the startup instead of the scratch.

L

Luca22500423wfl9

Participant

cr_sdk_

And no, this has not been solved.

To me, it happened generating 1:1 previews for a (very, very big) folder

Now my C: drive is stuffed by a 130GB temp file which is NOT TEMPORARY AT ALL.

Not restarting LRC or Windows deleted it. I will delete this by hand by it seems ridicolous to me.

Earth Oliver

Legend

are you sure that's not the Preview file that Lr generates?

steve.guides

Known Participant

Yes

S

SimLft

Participating Frequently

Same problem here.

Thanks for your very detailed posts FrankWPhoto.

When doing big pano, my 32GB of RAM is not enough and Lightroom start using my main drive as a backup for storing temp files in order to be able to achieve the pano.

My main drive is an old SDD connected by a Sata cable. I was hoping to find some solution here to tell lightroom to use my NVME drive (where lightroom is, the photos, the catalog) but it seems that's not possible at the moment.

So a NVME PCIE 4.0 for OS drive seems to be a good investment when doing a lot of pano (whatching out for overheating).

GoldingD

Legend

This being a old posting, and one that appears to perhaps not get absolutely resolved (think some misunderstanding going on)

I recommend that you create your own new posting for your problem/

When you do, include your issue, your workflow, and your system info as Lightroom reports it (first line thru and including plug-in info, /Help/System Info/Copy/)

However, some comments

These for WIN OS

1. The Camera RAW CACHE used by Lightroom Classic is not the same as the OS Paging File, or TEMP directory

2. The Location and size (limitation) for the Camera RAW CACHE used by Lightroom Classic is set in /Preferences/Performance/ (discussed somewhere above)

3. If at all possible do not place the Camera RAW CACHE on the same hard drive as the OS Paging File (Temp directory of no importance). If the Camera RAW CACHE and the Paging file are on the same hard drive, they will compete for read/wrights on that hard drive.

4. The Camera RAW CACHE should be, if available, on a very fast Hard drive.

5. The Camera RAW CACHE can be on any physical hard drive (but no, not a SD card, etc)

6. Your Catalog can also take advantage of hard drive speed, and can be on any physical hard drive (not a SD card)

7. Your photos can be on any hard drive, but their is no advantage as to hard drive speed (and as you can assume, not a SD card, after import)

OH, and remember, photos are not stored in the Lightroom Catalog, and when you back up the catalog, you are not backing up photos.

steve.guides

Known Participant

It's not just panoramas that cause this issue, either! When you publish a large batch of files (I use jf Folder Publisher to publish all my edited RAW photos into a set of jpgs for normal work use, slideshows etc). The publish process gradulally fills up the SSD system drive (using up 100gb of space that was free beforehand (i.e. 20% of the disk was free before Lightrooom started exporting). The publish starts off lightning fast and gradually slows as the drive fills up. Eventauly its so slow that I stop the task, shut Lightroom (at which point the SSD free space miraculously reappears) and then reopen Lightroom to start the publishing again where I left off. My workaround is to split the folders into smaller publish batches, but that really shouldnt be necessary if Lightroom let you set an overspill drive for temporary storage during large batch operations. It's odd that Photoshop allows this even though it rarely needs to handle large batch operations, whereas Lightroom is designed for handling large numbers of files.

P

pavelheinrich

Participating Frequently

Is it still unsolved? Did you find a way around it? I got pretty much the same scenario (only I have 37GB free from a 120GB OS disk) and a huge NAS, that I would love to use for a HDR panorama merge. I was watching the system to fill up without any of the temp directories to grow in size (at least not significantly). Btw FYI: on Mac the location of that temp dir in /var/folders is contained in the $TMPDIR variable.

F

FrankWPhotoAuthor

Known Participant

Hi, unfortunately, I did not. Although as I wrote earlier, it's been quite some time since I needed to make big HDR+PANO merges, so I do not notice it now in my normal work. But from your and earlier replies, it seems it still might the same issue. You are right about the location, I did mention TMPDIR, although if I rememeber correctly, it was nested further in $TMPDIR/Cleanup At Startup/[numbered folder]/cr_sdk_[some numbers].tmp (Cleanup At Startup is AFAIK an obsolete folder name used for scratch back in PowerPC Macs OS 9, and the numbers make it somewhat difficult to link the file somewhere else, and I am wary of linking inside system folders).

I am sorry I can't help you with testing it again on my machine right now, I can try to do a retest with some really big HDR+PANO merges later.

Although I can give you one warning - even if we did find a workaround or Adobe fixed the issue, you should never use a NAS for any kind of intensive file operations like scratch files - the slow network access will slow it down to a crawl. You could probably get by with an USB3/Thunderbolt attached external drive in a pinch, if you are on a laptop. I use fast (separate from system drive) internal SSD for big scratch files (although the problem was that Adobe couldn't use it even if I tried) on desktop.

D Fosse

Community Expert

Frank, you can go about this any way you like, but it all boils down to your 256GB system drive. That's a problem waiting to happen, if it hasn't already. It's just too small.

Lightroom doesn't need a scratch disk as long as you have enough space for the OS to do its normal paging.

The real problem is the way the user account accumulates and fills up on both Mac and Windows. Nothing is ever removed here unless you clean out yourself. An application uninstall just leaves everything behind, so it keeps accumulating without limit. Bits and pieces add up.

On Windows there's an excellent utility called WinDirStat that gives you a graphic presentation of what fills up the drive and exactly where it is. I'm sure there's something similar for Mac.

DdeGannes

Community Expert

What I was referring to is the 30GB of the 256GB SSD drive, that calculates to just under 12% and you risk loosing the disk completely.

Regards, Denis: iMac 27” mid-2015, macOS 11.7.10 Big Sur; 2TB SSD, 24 GB Ram, GPU 2 GB; LrC 12.5,; Lr 6.5, PS 24.7,; ACR 15.5,; (also Laptop Win 11, ver 24H2, LrC 15.0.1, PS 27.0; ) Camera Oly OM-D E-M1.

F

FrankWPhotoAuthor

Known Participant

Oh, alright, thanks. That’s a valid point, and I do usually try to keep the free space bigger than that for TRIM and SSD internal housekeeping. That’s why I was even more concerned about the behaviour, since I had set all programs to use the other scratch drive.

DdeGannes

Community Expert

Quote "Since there was a lot of the merges queued, my start-up disk almost completely filled up even though I had almost 30GB free on the SSD. I have dedicated SSDs in RAID just for scratch files and working with video, these are much faster than the start-up SSD (256GB) and had ample space. Is there any way to tell LR to use only those for scratch files? Mind you, these"

This is the root cause of your problem the 30GB is insufficient free space on the 256GB SSD drive. You should have at least 20% free space on the drive at all times when you go below that level you will begin to experience significant affect on your system's performance.

See the link below for the Adobe site.

optimize-performance-lightroom.html

This is a clip from the article.

Working with too little free hard-disk space can cause poor performance. Make sure that the hard drive that stores your Lightroom catalog, previews, and image files is at least 20% free.

See Lightroom system requirements to find out the minimum amount of free hard-disk space you need for your version of Lightroom.

Regards, Denis: iMac 27” mid-2015, macOS 11.7.10 Big Sur; 2TB SSD, 24 GB Ram, GPU 2 GB; LrC 12.5,; Lr 6.5, PS 24.7,; ACR 15.5,; (also Laptop Win 11, ver 24H2, LrC 15.0.1, PS 27.0; ) Camera Oly OM-D E-M1.

F

FrankWPhotoAuthor

Known Participant

Not helpful at all. Did you actually even read the thread? There was plenty of free space, and I had a several hundreds GB free ona second SSD, which was set as scratch drive for PS/LR. The bug is that PS/LR IS NOT HONOURING THE SCRATCH LOCATION. Same problem would manifest with 30% free on the system drive, with 90% free, just the same, only a little later in the multiHDR merge.

R

rizzlevizzle

Participant

The curious thing here on my end is that once Lightroom eats up all this space during a merge, it doesn't give it back to me after quitting Lightroom. The only way for me to get the space back is to restart my computer entirely (2013 MacBook Pro, 16GB RAM, macOS Mojave). I can't believe that this doesn't impact more people on a regular basis. Among Lightroom's zillion other problems, I'd say this is a pretty major one.

F

FrankWPhotoAuthor

Known Participant

So, from your reply I assume the bug is still there, even after half a year (I haven't done any major merging after noticing the issue, so I don't know)? Adobe memory/scratch management at its finest...:(

And yeah, it's only really noticeable if you are merging a lot of big photos into HDR/Panoramas, where the scratch grows rapidly into tens of gigabytes. Hiding the scratch in a hidden file in hidden folder on the system disk instead of respecting user-set scratch location is pretty bad.

S

simone_zini97

Inspiring

Have you tried creating an hard link from the directory you are using to where you want to store this kind of files? An hard link is like a normal link but it is used also from programs. I use it on Windows because I have a really small and pretty old SSD as C drive, only 64 GB, it is like a mirror, OS and programs think they are using it from C drive, instead everything is done where you want it to be redirected, in my case on my bigger and newer 750 GB SSD E: drive. I know it exists also on mac os but I don't know exactly how to do it, try do some research about creating hard links on mac os.

I also suggest you to check them sometimes because they might get corrupted. For example I moved my /Local folder on E: drive but when I update programs the hard link breaks and I have to rebuild it. For this i have written a little batch file that does the boring job for me.

It's pretty easy to notice if it breaks: the disk fills up . I also suggest a tool to discover big files and folders, on windows it is called WinDirStat but I know there are really similar programs for Mac OS and Android

F

FrankWPhotoAuthor

Known Participant

Hi, thanks, but see my reply above to pro360media why hardlinking it might be difficult/dangerous.

Hal P Anderson

Inspiring

You don't say whether you're on Mac or Windows. On Windows, at least, you can tell the system to keep your temporary files on a different disk. Type "move temp folder" into Cortana and go to the web page she suggests. There's probably a way to do it on Mac as well, but a quick view shows it's perhaps more complicated. Search the web for "move temp folder on Mac".

Hal

F

FrankWPhotoAuthor

Known Participant

Hi, thanks. Both you and JSM suggested moving the tmp folder. Alas, that's not so easy on Mac OS, and requires some low level tinkering. I am not even sure Adobe programs do honour the $TMPDIR variable or are just hardwired for the location. I might try that later, with a clean install.

Normally, for any application, it would be just fine to put any temp files in the system TMPDIR. But since ACR/LR scratch can grow up to tens of gigabytes very easily, and they both do seem to offer the user the option of selecting a "cache" or "scratch disk" location, it would make some sense to put the actual scratch file in that location as well, since the way it is now, it can be confusing what location is what. On the other hand, TMPDIR gets cleaned by the system on reboot (or the scratch file gets deleted by ACR/LR upon normal quit or upon ACR/LR restart if they crashed). Still, I would prefer if there was some clear documentation on that - could not find any at all, and had to parse through lsof output just to find what and where is eating up my free space.

Just Shoot Me

Legend

You can set the ACR cache location from the Performance tab of the LR Preferences.

Also purge the video cache. No matter what it is set to it just keeps filling up.

F

FrankWPhotoAuthor

Known Participant

Hi, thanks. I did that already, the ACR Cache location is set to my temp ssd raid array right from the start, did no difference. Nevertheless, that seems to be only the CACHE for storing PREVIEWS data, not an actual TEMP location. When doing a memory intensive operation like merging a lot of photos into a panorama, it still seems to eat just main system memory and that either gets swapped to (startup) disk by the OS, or LR itself stores the temp files in TMP or somewhere on the startup disk, disregarding the CACHE setting. Maybe it's temp files that get created right next to the catalog (which is currently stored on my startup disk), maybe it's just memory swapping but even if really so, it would be IMHO bad programming to rely on OS memory swapping only when potentially dealing with panoramas the size of hundreds of megapixels, or not offer the user a different location for the scratch files apart from system tmp, like Photoshop itself does.

I might try it again, do a lsof command on Lightroom, just to see what file operations the LR process does (lsof is an unix command showing all files opened by a process, supported on Mac OS natively).

Sorry, should have stated the system specs first - Mac OS 10.12.6.

Just Shoot Me

Legend

What exactly do you mean by Temp Location?

There are only 4 places LR stores info to.

1) The catalog file itself and that can be on any drive and in any folder you like.

2) The image file Previews (CatalogName Previews.lrdata) folder and that is stored (and has to be stored) in the same folder as the Catalog File.

3) The Smart Previews folder. If you use smart previews. And that also is stored in the same folder as the catalog.

4) The Video Cache which is stored someplace in your UserName folder and that can't be moved

All the others, other than the video cache, can be moved off the system drive, Windows the C: drive, Mac the Macintosh HD.

If somehow your Windows Temp folder, usually C:\Windows\Temp, is filling up then you can move that to some other drive using the System "Environment Variables". you have to create a Temp folder on some other drive first then point Windows to use that temp folder.

Make sure that you have a large enough hard drive and enough free space

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded