• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers

Creation of new Catalogue takes weeks, how speed up ?

Community Beginner ,
Sep 07, 2022 Sep 07, 2022

Copy link to clipboard

Copied

Hi

I have more than a million of photos. My old catalogue got corrupted and now I have to create a new Catalogue. To do this, I syncronize my files on Harddisk (NAS) with the catalogue.

Even when I have a very fast computer with a lot of RAM and a very fast NAS Drive, it takes days and days and after a while there is no progress anymore and I have to restart.

Probably the problem is related to build previews, but my settings are to build the minimum previews as possble. 

Can someone advice how to solve this problem ?

TOPICS
Windows

Views

132

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Sep 07, 2022 Sep 07, 2022

Copy link to clipboard

Copied

You stated "My old catalogue got corrupted and now I have to create a new Catalogue."

You have no recent backup of of your Catalog?

My estimate for what you are trying to do is 30 to 35 days.

 

 

 

Regards, Denis: iMac mid-2015, 5K 27”, GPU 2GB, Ram 24GB, HDD 3TB, macOS 11.7.1 BigSur,; LrC 12.0.1, Lr 6.0, Ps 24.0, ACR 15.0,; Camera OM-D E-M1

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied

No, I do not have a backup, It was too bog and backup failed.

It takes time with 'fetching initial previews'

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied

You could make manual backups, just copy the catalog file to another disk, using your operating system. Not quite as good as backups made by LrC (no integrity testing) but better than not having a backup.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied

I had copied it to another disk, but it did not work anymore

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Sep 07, 2022 Sep 07, 2022

Copy link to clipboard

Copied

I syncronize my files on Harddisk (NAS)

By that, you are referring to the file location for your images, correct?

 

and a very fast NAS Drive

The absolute best NAS is only as fast as your Ethernet, your LAN, Might be a super fast NAS, but it is still a dog compared to a local hard drive. Oh, while a lot better than storing them in the cloud, but..If you temporary copy the images to a local hard drive, does it perform better?

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied

Disk speed is only a small portion of the time needed to create a catalog. The larger bulk of time is creating the previews, which is done by the CPU.

 

One thing you can do to speed up the creation of the catalog is to select Minimal previews (as shown here, scroll down a bit). Of course, later after the photos are imported, you will see slowness as LrC generates larger previews when needed — its a tradeoff.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied


@dj_paige wrote:

Disk speed is only a small portion of the time needed to create a catalog. The larger bulk of time is creating the previews, which is done by the CPU.

 

One thing you can do to speed up the creation of the catalog is to select Minimal previews (as shown here, scroll down a bit). Of course, later after the photos are imported, you will see slowness as LrC generates larger previews when needed — its a tradeoff.



See my suggestion above. I believe that 'Embedded & sidecar' should even be faster than minimal previews, and it gives you previews that you can actually use for something more than just a grid view. That is why that option was introduced fairly recently.

 

-- Johan W. Elzenga

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied

I have already indicated "minimal previews" (now I changed it back to standard). It didn't help.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied

The time gets lost on the task "Fetching initial previews".

Copying all the data to my new NAS was much faster than creating the LR Catalogue from Pics which just need to be syncronized with the Catalogue, to be registered there. Any Idea how I can avoid the "Fetching initial previews" ?

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied

Wow, a catalog with more than a million images and no backup. I do hope you used to automatically write metadata to files, because if you didn't, then you have just lost over a million image edits too...

 

You could try to use 'Embedded & sidecar' for previews. With that setting, Lightroom does not create its own previews but uses the embedded camera-generated ones.

 

-- Johan W. Elzenga

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied

Yes, I always worked with XMP files instead of settings in the catalogue, thanks for the hint. I had tried this with the sidecars before, but it didn't speed up the import. Thanks for the idea.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied

You stated "Yes, I always worked with XMP files instead of settings in the catalogue, thanks for the hint."

You need to be aware LrC primary function is to write to the Catalog file, while you work in Lightroom it reads and writes to the Catalog file. As far as Lightroom is concerned the info is in the Catalog.

Writing metadata to xmp manually or automatically is a secondary function. Lightroom will read info from xmp at initial import thereafter that has to be prompted manually via the menu > Metadata > Read metadata from the file.
Saving metadata to the file allows sharing info with other applications, mainly Adobe applications Bridge / Camera Raw.

 

Regards, Denis: iMac mid-2015, 5K 27”, GPU 2GB, Ram 24GB, HDD 3TB, macOS 11.7.1 BigSur,; LrC 12.0.1, Lr 6.0, Ps 24.0, ACR 15.0,; Camera OM-D E-M1

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied

LATEST

Correct. You cannot 'work with XMP files instead of settings in the catalog'. That is not how Lightroom works. You can only save settings to XMP in addition to what Lightroom always does in the catalog. Stopping to make backups because the catalog was too big and Lightroom failed to back it up was still a very bad decision. Instead of giving up on making backups, this failure should have been a warning call that something was on its way to disaster, so backups were now needed more than ever! If Lightroom failed to make them, then you should have used something else. The XMP sidecar files will have saved you from losing the edits and that is good, but you could still lose quite a few things because not everything is written to XMP.

 

-- Johan W. Elzenga

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied

Can I ask how you work with LR Classic if you have two computers.

Especially if you have many pics which should be visible on both Computers and you cannot have the catalogue on a NAS-Drive (to be honest, I had it sometimes on a NAS-Drive with configuring a partition of the NAS like a local harddisk). Any recommendation for this ? Working with LR Cloud does not really work with so many pics....

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Sep 08, 2022 Sep 08, 2022

Copy link to clipboard

Copied


@Ann7 wrote:

Can I ask how you work with LR Classic if you have two computers.

Especially if you have many pics which should be visible on both Computers and you cannot have the catalogue on a NAS-Drive (to be honest, I had it sometimes on a NAS-Drive with configuring a partition of the NAS like a local harddisk). Any recommendation for this ? Working with LR Cloud does not really work with so many pics....


External hard drive. Keep format type in mind if MACOS and WINOS. External  can be a large RAID or simple single HD.  Lots of options on strategy as to where catalog is kept, including catalog copy's on separate computers.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines