Creation of new Catalogue takes weeks, how speed up ?

[Ann7]

Hi

I have more than a million of photos. My old catalogue got corrupted and now I have to create a new Catalogue. To do this, I syncronize my files on Harddisk (NAS) with the catalogue.

Even when I have a very fast computer with a lot of RAM and a very fast NAS Drive, it takes days and days and after a while there is no progress anymore and I have to restart.

Probably the problem is related to build previews, but my settings are to build the minimum previews as possble. 

Can someone advice how to solve this problem ?

[DdeGannes]

You stated "My old catalogue got corrupted and now I have to create a new Catalogue."

You have no recent backup of of your Catalog?

My estimate for what you are trying to do is 30 to 35 days.

[Ann7]

No, I do not have a backup, It was too bog and backup failed.

It takes time with 'fetching initial previews'

[dj_paige]

You could make manual backups, just copy the catalog file to another disk, using your operating system. Not quite as good as backups made by LrC (no integrity testing) but better than not having a backup.

[Ann7]

I had copied it to another disk, but it did not work anymore

[GoldingD]

I syncronize my files on Harddisk (NAS)

By that, you are referring to the file location for your images, correct?

and a very fast NAS Drive

The absolute best NAS is only as fast as your Ethernet, your LAN, Might be a super fast NAS, but it is still a dog compared to a local hard drive. Oh, while a lot better than storing them in the cloud, but..If you temporary copy the images to a local hard drive, does it perform better?

[dj_paige]

Disk speed is only a small portion of the time needed to create a catalog. The larger bulk of time is creating the previews, which is done by the CPU.

One thing you can do to speed up the creation of the catalog is to select Minimal previews (as shown here, scroll down a bit). Of course, later after the photos are imported, you will see slowness as LrC generates larger previews when needed — its a tradeoff.

[JohanElzenga]

---

@dj_paige wrote:

Disk speed is only a small portion of the time needed to create a catalog. The larger bulk of time is creating the previews, which is done by the CPU.

One thing you can do to speed up the creation of the catalog is to select Minimal previews (as shown here, scroll down a bit). Of course, later after the photos are imported, you will see slowness as LrC generates larger previews when needed — its a tradeoff.

---

See my suggestion above. I believe that 'Embedded & sidecar' should even be faster than minimal previews, and it gives you previews that you can actually use for something more than just a grid view. That is why that option was introduced fairly recently.

[Ann7]

I have already indicated "minimal previews" (now I changed it back to standard). It didn't help.

[Ann7]

The time gets lost on the task "Fetching initial previews".

Copying all the data to my new NAS was much faster than creating the LR Catalogue from Pics which just need to be syncronized with the Catalogue, to be registered there. Any Idea how I can avoid the "Fetching initial previews" ?

[JohanElzenga]

Wow, a catalog with more than a million images and no backup. I do hope you used to automatically write metadata to files, because if you didn't, then you have just lost over a million image edits too...

You could try to use 'Embedded & sidecar' for previews. With that setting, Lightroom does not create its own previews but uses the embedded camera-generated ones.

[Ann7]

Yes, I always worked with XMP files instead of settings in the catalogue, thanks for the hint. I had tried this with the sidecars before, but it didn't speed up the import. Thanks for the idea.

[DdeGannes]

You stated "Yes, I always worked with XMP files instead of settings in the catalogue, thanks for the hint."

You need to be aware LrC primary function is to write to the Catalog file, while you work in Lightroom it reads and writes to the Catalog file. As far as Lightroom is concerned the info is in the Catalog.

Writing metadata to xmp manually or automatically is a secondary function. Lightroom will read info from xmp at initial import thereafter that has to be prompted manually via the menu > Metadata > Read metadata from the file.
Saving metadata to the file allows sharing info with other applications, mainly Adobe applications Bridge / Camera Raw.

[JohanElzenga]

Correct. You cannot 'work with XMP files instead of settings in the catalog'. That is not how Lightroom works. You can only save settings to XMP in addition to what Lightroom always does in the catalog. Stopping to make backups because the catalog was too big and Lightroom failed to back it up was still a very bad decision. Instead of giving up on making backups, this failure should have been a warning call that something was on its way to disaster, so backups were now needed more than ever! If Lightroom failed to make them, then you should have used something else. The XMP sidecar files will have saved you from losing the edits and that is good, but you could still lose quite a few things because not everything is written to XMP.

[Ann7]

Can I ask how you work with LR Classic if you have two computers.

Especially if you have many pics which should be visible on both Computers and you cannot have the catalogue on a NAS-Drive (to be honest, I had it sometimes on a NAS-Drive with configuring a partition of the NAS like a local harddisk). Any recommendation for this ? Working with LR Cloud does not really work with so many pics....

[GoldingD]

---

@Ann7 wrote:

Can I ask how you work with LR Classic if you have two computers.

Especially if you have many pics which should be visible on both Computers and you cannot have the catalogue on a NAS-Drive (to be honest, I had it sometimes on a NAS-Drive with configuring a partition of the NAS like a local harddisk). Any recommendation for this ? Working with LR Cloud does not really work with so many pics....

---

External hard drive. Keep format type in mind if MACOS and WINOS. External  can be a large RAID or simple single HD.  Lots of options on strategy as to where catalog is kept, including catalog copy's on separate computers.