By my calculation, after migrating to a new external hard drive (due to a drive failure), Lightroom CC (4.0 MAS X64 on macOS) takes about 5 seconds to check 100 photos. I have a catalog of hundreds of thousands of photos.
After waiting 2 or 3 days for this checking process to complete, a move process appears to start... although underlying files have been moved already. So I'm a bit confused by this. That process eventually stalled for me. So I stopped it, and restarted Lightroom, only to see the *checking* process restart on the 290,000 files which had not been "moved".
So here are my questions:
- Why is this process so slow?
- Can it be optimized and cached so that it doesn't restart from scratch when Lightroom stalls or restarts?
- Why do files need to be moved if their relative position is identical on a different drive? Can't some in-place absolute path replacement be done in bulk rather than one file at a time, especially when the "Move" feature isn't about individual files but all of them.
Even with catalogs at a much smaller scale than mine, this seems like a very painful and unpolished process. Is there a chance the Lightroom team could improve it? It's not a common user need, but it's a fairly crucial one when it occurs, and it's an extremely negative experience — the app is entirely unusable for days (in my case most likely a week).
3
Replies
And speaking of the move process stalling. Here it is after two days stuck at 3230 of 290014.
After checking Console.app, it tells me what Lightroom should be telling me but clearly these errors aren't being surfaced in the UI:
Here's the error message so this thread is searchable:
NmbFileMover ERROR NmbFileMover ERROR Error copying file from /Volumes/Monolith/Lightroom CC/99b59acd32f340c29bce8f9b97331154/originals/2018/2018-06-29/4-2.heic to /Volumes/Apothiki/Lightroom CC/99b59acd32f340c29bce8f9b97331154/originals/2018/2018-06-29/4.heic: Unable to copy file.
This happens over and over again without any reaction from Lightroom. It's trying to do something it does not need to do. The files have already been moved from the "Monolith" volume to the "Apothiki" volume at the exact same location relative to the root of the drive. But I guess Lightroom can't handle that and expects to be doing the file moving, so gets confused and crashes when it can't find the files it expects on the drive they used to be on.
This is a bit of a flaw in the design of this system. Obviously, the "Monolith" volume no longer has the files. So, I'm guessing I need to rename the new "Apothiki" volume to placate Lightroom's inability to do a base file path replacement ("/Volumes/Monolith" => "/Volumes/Apothiki") on all stored file locations (or at least those that mention "Monolith"). Or I could delete all my originals and wait days for Lightroom to re-download terabytes of data from Adobe servers for files I already had safely stored locally. In the meantime, if anything happens to Adobe storage servers, years of my archives would be lost. That's a *lot* of trust to put in Adobe's systems when time and again similar systems have been known to lose or corrupt customer data. I hope Adobe engineers, however confident in their single source of truth architecture, understand how much of a leap of faith this is.
Regardless, I think it would be good for Lightroom to indicate in its UI these exceptions as: "Couldn't move file from X location to Y location, file couldn't be found at X location". That actionable message could help users in similar situations understand they've broken the relationship between Lightroom's database and local original files.
To document this even further, here's a video of how slowly Lightroom is checking files even after I've renamed my current storage drive to "Monolith" (to avoid the issue I faced in the previous reply).
https://share.getcloudapp.com/YEuQpP7G
I appears marginally faster than when I was originally running the process before but not noticeably. Importantly, while it's nice that Lightroom is being thrifty with resources when it comes to something that is likely to be running in the background, I'm surprised how little CPU usage or threads are dedicated to this process. I think I've seen one core peak at 30% or so on a 6-core system with hyper-threading and plenty of RAM to spare (1.95 GB use by Adobe Lightroom, 1.19 GB used by Adobe Lightroom Helper with roughly 10 GB free).
Update, the checking process finally completed. Only to result in this:
No idea how to move forward. "Monolith" is absolutely not "offline". But knowing how Lightroom occasionally fails to surface file permission issues, maybe it's missing some permissions. I don't know...
I have no idea what the "Forget Volume" button would do, or what consequences this would have. Why are 1464 photos unable to be moved? Did all 300K+ others move successfully, or were they successfully found on the hard drive that is supposedly offline?
AdChoices