Skip to main content
Known Participant
May 1, 2011
Open for Voting

P: Allow Catalog to be stored on a networked drive.

  • May 1, 2011
  • 559 replies
  • 13726 views

I'd love to make LR more multi-computer friendly. I have no doubt that there's probably database architecture issues and a host of other barriers... But I have to believe that the need for either multi-user or at at lease multi-computer use is widely desired. And yes, I know you can do the catalog import export thing but I find this less than ideal.

559 replies

Inspiring
May 15, 2023

However, I recently changed my workflow when I began using LrC v12.3 to process timelapse photos into videos, and have found that LrC becomes unresponsive while importing 400+ photos when both the photos and catalog reside on the NAS. As a result, I've had to kill LrC quite a few times, leaving behind a .lrcat-wal file with the changes that haven't been applied to the catalog. Restarting LrC automatically recovers these, and the catalog passes an integrity check.

 

I've run some tests, with the following results:

    Photos: NAS, Catalog: NAS, Previews: NAS: Fails
    Photos: NAS, Catalog: NAS, Previews: SSD: Fails
    Photos: SSD, Catalog: NAS, Previews: NAS: OK

 

Thus, it's not clear that the problem is directly related to storing the catalog on a network drive.

I just tested:

    Photos: NAS, Catalog: SSD, Previews: SSD: Fails

So this appears to be an LrC bug, and not related to storing the catalog on a network drive.

 

Inspiring
May 15, 2023

Several years ago, LrC switched to accessing SQLite in WAL (Write-Ahead Log) mode. According to the documentation, WAL mode limits database access to the (LrC, in this case) processes on a single PC, and these processes coordinate through shared memory. I used several tools (watch, inotifywait, wireshark) to observe catalog operations while having LrC sync develop settings across >3K photos and found that other than the initial oplocks, no other lock requests are sent to the server. SMB/Samba implementations with file locking defects, should there still be any after all these years, shouldn't be an issue preventing a catalog from being stored on a network drive, as I've been doing since 2011 using the method described here.

 

However, I recently changed my workflow when I began using LrC v12.3 to process timelapse photos into videos, and have found that LrC becomes unresponsive while importing 400+ photos when both the photos and catalog reside on the NAS. As a result, I've had to kill LrC quite a few times, leaving behind a .lrcat-wal file with the changes that haven't been applied to the catalog. Restarting LrC automatically recovers these, and the catalog passes an integrity check.

 

I've run some tests, with the following results:

    Photos: NAS, Catalog: NAS, Previews: NAS: Fails
    Photos: NAS, Catalog: NAS, Previews: SSD: Fails
    Photos: SSD, Catalog: NAS, Previews: NAS: OK

 

Thus, it's not clear that the problem is directly related to storing the catalog on a network drive.

Anyway, processing timelapse photos residing on the NAS with LrTimelapse isn't practical, since it's far too slow.

johnrellis
Legend
May 13, 2023

I find I have to summarize the issues every couple of years to avoid mistaken information from propagating:

 

LR uses the widely used open-source database SQLite to implement its catalog.

 

To avoid corruption when a database is accessed concurrently, SQLite relies on the file system to correctly implement POSIX or Windows file locking.

 

In years past, circa 2007, many NAS implementations, including the then-popular enterprise NFS, didn't implement file locking correctly. The SQLite authors reported having been told that some Windows SMB implementations didn't either, but they didn't provide any authoritative references.

 

Around 2007, Adobe employee Dan Tull tested LR with a catalog stored on a Mac SMB share with a Windows XP client, and the catalog got corrupted when he unplugged the network cable while LR was running. 

 

Early versions of LR had a config-file option to allow catalogs on NAS, but that was removed many years ago.  Regardless of whatever warnings are provided to users, Adobe was afraid that there would be too many users who would ignore the warnings and then come to them to repair their corrupted catalogs.

 

These days, I think nearly all NAS owned by LR users is based on SMB, Microsoft's protocol. Microsoft has their own implementation, Apple has theirs, and I believe most everyone else (including NAS vendors) uses the open-source Samba.  Everytime this issue rears up, I search for current information about the reliability of file locking on these SMB implementations, and I never find anything new.

 

Since Dan Tull's post years ago, I've seen no other information from Adobe suggesting they've retested LR Classic on commonly used NAS.   At this point, I doubt that Adobe would ever put the effort in, given their focus on LR Cloud and their publicly stated reduced effort on those parts of LR Classic not related to Develop.

 

 

Participating Frequently
May 11, 2023

The only practical way to deal with this - is to split the catalog from the data files. Put the catalog on the local drive and the data on the NAS. Thats it. Don't waste ONE moment of time trying to get the catalog to work over NAS - it will not. You can spend three or four hours reading this entire chain of emails - to find out: Adobe does not support Catalogs on NAS.

Throughout this chain the basis for the reasoning the capability to use network attached storage is that SQLite is not capable of supporting transactions is supported with a 15 year old internal study.  While it might still be true for Data Center spans (even less likely), Cloud or Public networks (likely) - it is simply not true for lans, particularly with the two devices connected to the same switch. The idea that SQLLite does not support database transaction semantics today - is also now very false. 

For consumers - there is no practical difference between direct attached storage (examples are: Thunderbolt Drobos and Pegusus Arrays) and the network attached storage of a Synology.  Adobe is being silly, and we should all consider this to be a marketing restriction. My own implementation of NAS is better than any retail DAS... but it simply doesn't matter. Said more practically - just get over it - Adobe has demonstrated for more than a decade that it is unwilling to lift this restriction. Since Adobe will not even offer a reasonable explaination - we must assume Adobe is not going to change this restriction. That is Adobe's decision to make.

My 5 TB of photos are now on two geographically distributed synology systems, with a local physical backup on both (sync is not the same as a backup), cross linked with file versioning, switch connected optically with multi-link 10G.  I built this for completely unrelated reasons, but it is more reliable than my previous DAS gear - Drobos, and then two Pegasus R6 arrays. FWIW - the R6 arrays are fantastic and I remain a huge fan (don't use AFS on them), but Apple's declining support for any kind of server/professional/enterprise anything made the use of those systems obviously impractical, ergo the switch to Synology.

10 years ago, Apple started to divest itself of any semblance of support for direct attached storage in the form of disk arrays - see if you can find any system admin tools any longer... They completed that work several years back when the system was forced onto its own partition in the implementation of AFS.  Great reasons for it - but the penalty was that Apple Software RAID really just doesn't work - which I learned the hard way when drive reconstruction - didn't reconstruct. Twice.  Because of the direction that kernel drivers are going to support security - don't be surpriseed when SCSI and other drivers needed to run a DAS - do not work. 

Hence - the only pratical way to deal with this problem - is to split the catalog from the data files / working directories. Put the Catalog on the local "Data" partition - and take pains not to purchase the smallest Apple internal drives.  The conversion to the split catalog / data takes a little doing - but it works, and it is supported by Adobe.  Many of us have a workflow which is to create a catalog for every working set, that works too - its just a bit more setup.  If you keep separate catalogs like that - don't try to copy them to/from Local Storage <--> NAS.  You _can_ - but you are much more likely to make a mistake and lose something. 

There are other applications which do 50 to 80% of what LightRoom does. I've tried many. None came close to matching the efficiency of the LR workflow capabilities.  So - maddening as it is - I stopped fretting over it, and made it work for me.

Conrad_C
Community Expert
Community Expert
May 11, 2023
quote

Synology DS1823xs+ NAS…How could I have known that LR Cats cannot open on a NAS?…tips on a new workflow or work arounds to make this work.
And if anyone at Adobe could tell us why Catalogs cannot open off of a NAS, that would be wonderful.

By

 

The specs for that NAS say it has three USB 3 5Gbps ports. Is it possible then to simply use it the same way as the Drobo, as a DAS connected by USB? That would get around the network limitation.

 

The explanation about why catalogs are not supported on network drives is earlier in this thread. Although there are many saying it ought to work, in fact many other DAMs use the same SQLite and have the same limitation. I think Apple Aperture was this way too. It takes a real enterprise level DAM designed for networks to get around this, and although I am no export on them, those seem to be in a whole different price bracket than these consumer DAMs.

Participant
May 11, 2023

This lack of support for NAS (or even the cloud) is just one of the many reasons Lightroom becomes more and more irrelevant as time goes on.

Participant
May 11, 2023

Hey all. 
I am a photographer of 15 years and I've been using LR for most of that time. My workflow to date has been to work off of a Drobo Direct Attached Storage RAID system. but now that Drobo is bankrupt and no longer supports firmware updates I have been forced to buy a new system to keep my iOS and all other software up to date. 
After much research, I went with a Synology DS1823xs+ NAS. I just set up the system, got all of my data copied over into the new drives, double-clicked one of my LR catalogs so I can get back to work, and BOOM: "Lightroom Catalogs can not be opened on network volumes, removable storage, or read only volumes."
This is news to me! How could I have known that LR Cats cannot open on a NAS? 
So I am looking for some advice on how to find a solution. Either some tips on a new workflow or work arounds to make this work. 
And if anyone at Adobe could tell us why Catalogs cannot open off of a NAS, that would be wonderful. Thanks for the help, all! 

 

Participating Frequently
December 5, 2022

It's sad that it's been over 10 years since this request was posted and it still hasn't been implemented. PLEASE ADOBE!

Participating Frequently
December 5, 2022

Hi, I've read a few posts about this and understand it's intentionally not allowed but I think this design needs reconsidering. I work with multiple people and computers and we use the same data stored on a network volume (specifically a Synology NAS). Lightroom is the only software I've run into that doesn't like this and it's a huge inconvenience. This working environment is common and a lot of people will be driven to other photo editing packages.

 

PLEASE! Enable Lightroom Classic Catalogues to save onto network volumes so that the catalogue can be edited on different computers (even if not at the same time) without needing to copy locally first.

 

Thanks for your consideration!

johnrellis
Legend
December 5, 2022
Participant
June 16, 2022

I am an avid Lightroom user and a marketing director for a small real estate firm. We need to process and archive thousands of photos a year, and I don't have a good tool for this. Even with a small team, we don't have a good solution for rating, batch processing and archival of photos. I would hate to have to manage this across a large team. Does anyone have any idea how I should be doing this? Looking at the Adobe product set, I'm frustrated. Lightroom is not keeping pace with modern workflows.

 

  1. Lightroom Classic is not able to be used collaboratively across teams in a professional environment. Want multiple retouchers working within one catalog? That's impossible! Want a master catalog that a marketing department can use as a digital asset management system? Too bad! I'm tired of not having an adobe-based system for photo editing, storage and collaboration. This is a glaring oversight on Adobe's part.

 

2. Adobe Lightroom CC has a completely hamstrung feature set designed for toddlers compared to the power behind Lightroom Classic. I can't even batch rename photos which is one of the most important parts of my workflow.

 

3. The best solution for using lightroom collaboratively is to have a separate machine with the catalog that is accessed by remote desktop. FAR from ideal especially when you're dealing with extremely high-resolution photos which just slows down the remote desktop process.

 

4. Because it's such a pain to collaborate I have to be the one person processing photos on my team which is far from an ideal workflow.

 

Who has solved this problem and what tools are you using?