P: LRC - system drive used at 100% - process identification needed

[Luke PhotoArt]

Dears - some time ago I've posted a similar topic, which as I thought was solved - but as it came out, not completelly. I still need your hep / support - as currently it is so annoying I'm thinking of changing the platform 😞 Sorry for the lenght of the thread - but everything needs to be as clear as possible 😞

In short words - the original thread is here 

https://community.adobe.com/t5/lightroom-classic-discussions/lrc-uses-100-of-system-drive-while-edit...

Just to be clear with sytem specs:

Windows 11

Ryzen 9 7950x

Radeon RX 7900xt

32GB RAM DDR5

System drive NvME 256GB Kingston - where LRC and PS are installed. about 70GB free disk space (new firmware installed)

Drive for RAW files, and a lrc catlogue is a separate SSD, connected via SATA (2TB Total), cache (100GB). So basically all oferations are done not on a system drive, but on a separate SSD drive.

All options of performance tab using GPU checked for full support.

LRC always at latest version - but the issue persists for several months already - so version doesn't change anything.

ISSUE is: During photoediting in lrc, the software freezes for 2-3s (randomly - when changing photo / adjusting exposure / while applying masks) - in general no role as of when, and why. During that freeze task manages shows sudden read / write operations on system drive (where the software is located), that takes up 100% system drive capacity - spikes that last for few seconds (task manager show 250-300 MB/s ssd read / write activity) - after which the usage goes back to 2-5% and the soft runs smoothly.

It was suggested that it might be spotify running in the background, or windos defender. But adjusting options / disabling did not solve the issue permanently - it still occurs.

THE DEVELOPMENT of finding the issue. I've installed "process explorer" to try to find the thread, or a process that runs during this issue. And althought I did not find out if there is any file read or wrote during the 100% drive usage, I found out, that along theese freezes there are 3 main dll files being suspicially overactive (they ofen pop out as CPU most active threads under lightroom.exe process freezes). Theese are:

ntdll.dll
uctrbase.dll
substrate.dll

Can it be, that theese libraries are the main cause of the issue? What are they responsible for - is it possible to identify a solution, based on theese 3 files activity?

Any help will be apreciated - if I can't solve the issue I am afraid I will switch to alternative software - as I can't work in such conditions. 😞

[Sameer K]

Hey again, @Luke PhotoArt. I'll continue to help you figure this out. Great effort, investigation, and summarizing what you found. 

These are critical Windows OS components, so it may be an OS-level issue. A quick web search suggests methods to repair the OS image. Here is what I found to repair System Files

1 - Run the following commands in Command Prompt (Admin):

sfc /scannow

DISM /Online /Cleanup-Image /RestoreHealth

You'll find relevant help docs to troubleshoot and follow these steps on the internet. You'll have to approach this broadly. 

Also, try these steps in Lightroom Classic:

• Lightroom Classic > Catalog Settings > Metadata > Uncheck 'Automatically Write Changes Into XMP.'
• Check this detailed response from our Expert: https://adobe.ly/4g6Cb4j

Let me know how it goes. Thanks!

Sameer K

(Type '@' and type my name to mention me when you reply)

[Luke PhotoArt]

Also, try these steps in Lightroom Classic:

• Lightroom Classic > Catalog Settings > Metadata > Uncheck 'Automatically Write Changes Into XMP.'

Hi @Sameer K  - hope this time we can get it solved fully :

Just one question - I have this option already unchecked - shall I check it then?

[Sameer K]

If it is unchecked, we can proceed past it. You can go to File > New Catalog and use a new catalog to test if the issue exists. 

Thanks!

Sameer K

[Luke PhotoArt]

@Sameer K I've followed this solution

sfc /scannow

DISM /Online /Cleanup-Image /RestoreHealth

No change. 😞

As for "Check this detailed response from our Expert: https://adobe.ly/4g6Cb4j"

It involves only cpu usage - which doesn't reflect my issue which is system disk usage 😞

I have also tried, to make new user accound in win 11, without any background processes, aditional software etc... just lightroom. Still the issue persists 😞

I've attatched a film screen recording, that shows task manager, and process threads at the same time. Pls look, when it spikes to 100% - write data nearly 300MB/s, C drive response time over 1000ms 😞 It is like something is being constantly written on system drive - is there a way to check, what files are being written at this moment?

I am also thinking about the catalogue size, as mine consists of photos from 5 year, but it's only 18k photos in the catalogue, plus the catalogue itself is on a different drive, which isn't affected.

Any advices more? I was thinking of mooving whole adobe package to another drive - but it seem I cannot do it without uninstalling LRC and PS - which means loss of LUT's, presets and custom settings - which would be a huge issue for me 😞 Unless you know a way, to move adobe software to another drive, and keeping all the personal things?

[Sameer K]

Thanks for the updates, @Luke PhotoArt.
Q - Have you tried creating a new catalog as I've suggested above? 

Q - Does the spike appear after a specific set of development operations? 

While not currently needed, you should know this for future reference:

1 - You can back up your presets for Lightroom & any other presets for Photoshop.

How do I migrate presets for Lightroom

2- Backup preferences and presets for Photoshop: https://adobe.ly/4ipiCq4

Thanks!

Sameer K
(Type '@' and type my name to mention me when you reply)

[Luke PhotoArt]

@Sameer K 

Q - Have you tried creating a new catalog as I've suggested above?  - Not yet, unfortunatelly I have an urgent work to be done in lrc and simply have no time for such trials 😞 Will try that, when I finish urgent work - and then I will try to create new cataloge.

Q - Does the spike appear after a specific set of development operations? - none specific. Most common are:

1. Photo change in development tab (just lef / right key arrow)

2. sliding exposure slider

3. selecting existing mask, and / or trying to fine tune it but adding / removing with another mask

4. Sometime while cropping - but this is most rare.

As I do most editing via synced adjustments (have one photo done with colour / basic adjustments / masks (most using AI selection) - and synced to other photos - around 800-1000k other files), and then navigating the other files, and just fine tuning them. Which mean every new photo I open in develp tab, already has several masks applied, and other adjustmens already made by deafult.

Will study the backuping topics when I finish urgent work. Then I will consider moving adobe package to another drive. Thx for links.

[Luke PhotoArt]

@Sameer K And pls remember - that all catalogues, raw files, xmp files etc... are on a separate ssd drive which isn't affected at all. C: drive that is the issue with the spikes - it is the place where LRC and PS are installed to.

It looks like while editing photos located on a different drive, for some reason there are some LRC assosiated processes, write / read on C: which cause LRC to freeze for few seconds. 😞

[Luke PhotoArt]

@Sameer K And one interresting observation - theese are not CPU realated issueas at all. As during spike cpu usage is at 4-5%, which is normal usage, as well as during normal work. Also the threads as shown in video, use on 1-2% of CPU consumption. So CPU is not a bottleneck here 😞 Neither is GPU as it barerly used during theese spikes. Mem is usually at 50-60% used of all 32GB 😞

[Luke PhotoArt]

@Sameer K Took the liberty to record a screen with taskmanager on the side, to visualize the freezes I am facing - in this instance they start from around >50s of the movie.

One last idea from the top of my head. AMD adrenalin software - could this impact the performance?

[Sameer K]

Hey Luke! Thanks for continuously sharing what you've found.

When working with large sets of images and multiple masks, bulk changes are cached, thumbnails are processed, and the results are stored in temporary locations. Based on the scale of your workflow, the read-write operations are expected. 

To verify this further, you can try a new catalog with a scaled-down workflow (less copy/sync or smaller batches) and check if the data chunks appear smaller. 

In Lightroom's Preferences > Performance, you can increase the total Camera Raw Caches. Also, not to sound like a broken record, but I'd recommend you follow the content here to fine-tune settings across the Operating system and Lightroom on both hardware and software sides: https://adobe.ly/488DApC;

This document has three sections to review, so please take your time. 

Thanks!
Sameer K
(Type '@' and type my name to mention me when you reply)

[Luke PhotoArt]

@Sameer K Thx for all your effort - apreciate it.

I will try to work on a new catalogue when I finish my current project.

Sure will read the other topic, and try to adjust the parameters.

Btw. The cache is already 100GB and it's stored on other SSD, then the one that couses problems - so in this case I do not want to sound like a broken record 😉 But I wouldn't connect theese two factors.

Also - as for working on smaller batches of photos. I wouldn't say I'm above standard or something. Ex - when you go for a wedding shoot, you bring 2-3k raw images and all of them need processing the same style - which means the invention of "sync" in LRC is more then necessary. Unfortunatelly it involves AI also - as I do not imagine editing photos one by one withouth it. In my opinion if theese features are introduced they should be used - by definition to ease up workflow. Won't you agree? Besides - LRC works on XMP files, which are smaller, even if you consider previews files, still the size is considerably small. I would agree here, if my system specs were budget and low quality ones - but they are not. 😞

I will still stick to one thing, which doesn't add up with your way of thinking - All catalogues, xmp, photos, preview and all essential data is stored on a drive which is not affected. So even if the workflow amount can make the system overloaded it should impact mostly RAM and CPU, and the SSD where files are stored - which are not even used at 25-40%. And the issue persists on system drive read / write operation - what operations can be done there, if there no essential files there - just adobe package installation folder? In my opinion this is the question we should answer. Is there a way to check what specific operation is done during this write operation on system drive?

[Luke PhotoArt]

@Sameer K Hi. As you suggested, I've created a new catalogue, with only 2k raw files - basically double of the collection I've been working on. Unfortunatelly no change in spikes - still happening. New observation though - LRC can work for 1h straight without any issues. And suddenly - (working conditions haven't changed in the meantime) - it starts freezing with c: drive spikes for as long as 5s with "lightroom isn't responding" - untill it unfreazes again. I am sorry, but it simply cannot be a question of catalogue / optimalisation etc... If it was, the problems would be consistent - they are not, and seem not to be connected with anything logical (no new windows process, no change in LRC parameters - nothing!) I will read the thread of optimalisation, and will search for any possible solution - but I strongly doubt it. I am having this issue for nearly 6 months now, and honestly with each new LRC version, the issues seem more problematic - I am fed up with that - I wanted to avoid it, but now I am determined to trial test other alternative software, as my LRC licencense ends 23rd of Dec.

Yes - right now I am on the edge of my patience. As I've worked on 2k raw files 30% more time, then I should have if LRC was not causing any troubles. 😞

[Luke PhotoArt]

@Sameer K I've revised the topic for CPU usage - none of the solutions worked. 

As for "Also, not to sound like a broken record, but I'd recommend you follow the content here to fine-tune settings across the Operating system and Lightroom on both hardware and software sides: https://adobe.ly/488DApC;" It seems the link is broken.

Anyhow - I've done last resort action - I've reinstalled LRC to another drive (but kept settings and prefferences during uninstalling). So far it works flawlessly - the issue is gone, and everything seems to be running more smooth. System responsivness is higher, actions are done faster, and what is most important - file export is not way quicker then previously. I will continue to test it further. I'll post update later this month when I do more testing.

[KR Seals]

What is the size of the drive where LrC is now installed? 

After reading this thread, I am wondering if a new much larger system drive, like 1TB, would solve the issue? It's really easy to upgrade/mirror the system drive using something like Samsung Magician with Samsung drives.

[Luke PhotoArt]

@KR Seals 

What is the size of the drive where LrC is now installed? 

After reading this thread, I am wondering if a new much larger system drive, like 1TB, would solve the issue? It's really easy to upgrade/mirror the system drive using something like Samsung Magician with Samsung drives.

By @KR Seals

The size of SSD with new LRC installation is 2TB total - around 30% free disk space. Good question if new larger ex 1TB system drive would solve the issue - I will not take that chance to check XD The issue I had had already cost me too much time and nerves 😉 investing money in a new 1TB NvME drive just for testing purposes is not my goal 😉 I treat system drive as a base for software only + win - do not keep anything else there. That's why I use 256GB NvME - which should be the fastest option in theory. But in reality - something has failed. Sad that Adobe claimed "optimalisation" issues constantly, as it seems clear it's a software issue 😞

[KR Seals]

My system is setup the same. One drive is for the OS and programs. All data, photo and video files are on separate SSDs or HDs. Your issue with the system drive filling up is not a common one in this forum.

Larger SSDs are generally faster. My system drive is a NVME 1TB and it's less than half full.

[Luke PhotoArt]

My system is setup the same. One drive is for the OS and programs. All data, photo and video files are on separate SSDs or HDs. Your issue with the system drive filling up is not a common one in this forum.

Larger SSDs are generally faster. My system drive is a NVME 1TB and it's less than half full.

By @KR Seals

Can you do me one favour? Since you have the same setup as me - as one thing more bothers me. I have a full GPU acceleration enabled (also with exporting files). And currently when exporting it seems like gpu usage is around 10-15%, cpu usage is around 20-25% - but system drive again is 100% occupied (reminder - no LRC files there, neither cache for LRC). Can you check if it's the same instance with your setup (like you would run taskmanager while exporting a batch of files?

[Luke PhotoArt]

I've just dicovered an interrsting observation. When I turned off GPU acceleration for exporting, the system drive is at 5-10% usage max, and CPU at 90-95% (which is ok as for my understanding). 

Which means - if GPU export acceleration turned on (GPU usage not even at 20%), system drive is used at max capacity with write operation during export (even though no LRC associated files are located there). And this is the question for Adobe tech experts @Sameer K is it normal?

[KR Seals]

Test export of 30 45MP image files. CPU peak 46%, average 20%, GPU 13%. Zero activity on C: (OS) drive. 

[Luke PhotoArt]

Test export of 30 45MP image files. CPU peak 46%, average 20%, GPU 13%. Zero activity on C: (OS) drive. 

By @KR Seals

@KR Seals Thx man - apreciate it. Just to understand this is with GPU export enabled?

[KR Seals]

Yes...

[Luke PhotoArt]

@Sameer K Need your help, as something happened again. Yesterday during LRC usage, the system froze at some point. Had to manually restart it. After that I realised all my GPU info was gone (system stopped recognizing it). Had to reinstall AMD Adrenaline software. As a result LRC performance went to hell again - all the issues returned. But final observation is:

When GPU acceleration enabled - system drive is occupied both also when standard work (C: drive usage 100% again, even though no files are on C: drive), as well as during exporting files. Which as KR Seals proved is illogical - as his setup is free of this bug.

When GPU acceleration disabled - no issues at all C: system drive remains idle at any point of LRC usage.

So the solution to my issues seems logical - it is a RX 7900XT with LRC issue (btw. all AMD optional features are turned off, when it comes to LRC). But why? I've read reports over net, that AMD sometimes gets in conflict with LRC, but is it really possible? I am hoping I can get a definitvie answer this time.

[Luke PhotoArt]

@Sameer K It is sad, that noone from Adobe side reacts anymore - giving any advices. I am trully dissapointed.

Anyhow - maybe someone will this helpfull at the end. Just tested if changing the GPU to RTX 5080 will change the situation. And here are the conclusions:

RTX 5080 studio drivers installed - , and here's the conclusion: 

1. Overalperformance of LRC is noticably better.

2. During photos editing, system drive is no longer showing spikes - it remaind idle, which is good behaviour.

3. But during export with GPU acceleration, the issue continues - system drive is bufforing as previously at 100% usage

Temporary solution I applied is to disable GPU export acceleration, and push all process to CPU - it handles it well, then system drive is less stressed - at 2-5% usage. So it's not a problem.

Anyhow it seems that despite solving one issue of editing freezes (with system drive overloading), during export it still for some reason uses system drive at 300-700MB/s writing operations.

Conslusions:

For some reason AMD GPU doesn't work well with LRC and it seems Adobe realises that issue, but they they remain tight-lipped for that topic - as there are many topic like this on reddit - that confirm that with different drivers, similar issues of LRC freezing, and getting stuck are common

.

The issue of using system drive for exporting while GPU export enabled is left insolved - if someone has similar problem, I reccomend turning export GPU acceleration off - if your CPU is decent. Found no solution for that neither.

[Sameer K]

Hey, @Luke PhotoArt. Thanks for sharing the detailed findings. I am sorry for the delay in getting back to you. 

There's a lot of specific information shared in this thread. I'll work on sharing this with the product team to review.

Also, two things: have you tried a different/new catalog yet?

Lightroom Classic v15.1 is available for update. You can update, and let me know if the behaviors still exist. 

Thanks!
Sameer K
(Type '@' and type my name to mention me when you reply)

[Luke PhotoArt]

Hey, @Luke PhotoArt. Thanks for sharing the detailed findings. I am sorry for the delay in getting back to you. 

There's a lot of specific information shared in this thread. I'll work on sharing this with the product team to review.

Also, two things: have you tried a different/new catalog yet?

Lightroom Classic v15.1 is available for update. You can update, and let me know if the behaviors still exist. 

Thanks!
Sameer K
(Type '@' and type my name to mention me when you reply)

By @Sameer K

@Sameer K 

Yes - tried new cataloge + new drive - no change.

Update for LRC - I am just downloading, I will share the results.

Btw. Did some tests with process monitor application. On two almost identical setups (already with RTX 5080 installed, as a replacement for RX 7900XT).

Test conditions. Exporting the same batch of files, with exactly the same settings, but one based on GPU export, second solely on CPU. The result are clear - CPU, no system drive noticable usage - 2-5%. BUT with GPU enabled constant spikes with writing operation up to 800MB/s. So I checked, and recorded what files are being written with lightroom.exe on system drive. And the results are interresting - it's like constant hundreds of times per second write, then rewriting operation proceeding, on and on again and again - only on cr_sdk_.....tmp file. I even cought this file once, to have 5GB size (screen attatched) And while on CPU export this file is also active - it's like 10% volume comparing to GPU export. 

If you want I can share the .pml file (process monitor application file) with records of files usage from proces monitor. All lightroom.exe operations on system drive are recorded there, during export.

Hope it helps.

I can also share LRC logs - as I've noticed such are created - maybe this'll shine some light.