• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
0

Lightroom Classic - import progresses till certain point and then ... no real progress

Community Beginner ,
Mar 20, 2021 Mar 20, 2021

Copy link to clipboard

Copied

It is Lightroom Classic (not CC) version 10.2, Camera Raw 13.2

I'm rebuilding my catalog.

In a nutshell I am importing relatively big number of photos (150k+) into the catalog. It is a fresh import.

The behavior is the same every time I start the import:

Import starts, it progresses to some point and then no further visible progress.

Looking on the disk’s activity - nothing happens on the disc/network side.

Catalog seats on separated nmve SSD with sufficient disk space (400GB+ free, system SSD has 300GB+)

System: i9 CPU; 32GB RAM; Windows 10; Images are on the NAS with fast dedicated network.

Lightroom manages to import portion of images and then stops doing anything. The portion could be 1000, or it can be 15.000 images at once. Leaving the system for few hours, or 1,5 day – the result is the same: portion is imported and then it stops doing much.
When this stage is reached, Lightroom power and CPU consumption goes down to ~2-4%. Usually it happens when Lightroom occupies 15-18Gb of RAM.

The lightroom has not been able to import all images at the single run.

It is easier to interrupt (pressing the red square next to the progress bar on the top left side of the screen) and restart the process - this gives much higher "number of images per hour".

Any idea why this happens? What could be done to improve the process speed and stability?

 

TOPICS
Windows

Views

673

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Employee ,
Mar 26, 2021 Mar 26, 2021

Copy link to clipboard

Copied

Hi there, 

 

We're sorry for the trouble. Have you tried importing files from the internal drive? Do you have the files on the same SSD drive? Also, are you seeing this issue after the recent update of Lightroom Classic?

 

Could you please try an import a small number of files in one go. Try 50 images and let us know if that helps. 

 

Thanks,

Mohit

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Mar 26, 2021 Mar 26, 2021

Copy link to clipboard

Copied

Hi Mohit,


I have the latest version, at least when I have checked an update earlier today - it was on the latest available.
If I import from the folder with 177 images - it worked fine.
this was an import of the remaining images from one of the "incomplete" folders (at least there were some not imported images there) - it could handle this number within very reasonable time (it took less than a minute) and with no delays, and most important, the import has been finished completely.
In the former time it was hanging at some percentage, e.g. 77%, or 81, or could be 90%... and the progress bar on top being not progressing.

 

I'll try to import bigger portion, let's see.

 

Regards,
Andrey

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Jun 27, 2021 Jun 27, 2021

Copy link to clipboard

Copied

The problem with Lightroom import persists.

Multiple updates have been applied to the lightroom classic already – issue is not fixed.
It looks like internal process related to the import gets crashed (guessing), otherwise it is difficult to explain this behavior.
The pattern is the same: the import starts, all goes “fine” until undefined moment, when the import stops showing no further progress.
Progress bar on the top left corner stops somewhere, initially during previous import activities, it was visually frozen at 75-80%, but today it is at ~30-45% and shows no further progress.
Today I try to add (with copy) a couple of images from the card.
From yesterday’s session 67 out of 135 pictures were imported, and from today’s session it is 6 out of 78 images.
Lightroom consumes about 13-14% of CPU load and is the only one process with very high-power consumption status. At the same time, it does not show any activity neither with disk nor with network.

This issue becomes way too annoying.

Can you please provide a fix for this problem? At least have some meaningful message that something does not work properly.

Thank you in advance,
Andrey

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jun 27, 2021 Jun 27, 2021

Copy link to clipboard

Copied

Try importing the photos in smaller groups. Instead of all at once.

 

Do you realize that "rebuilding your catalog" by re-importing everything causes some information in your catalog to be lost?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

I tend to keep the settings synced to the file system, that it is available for other applications if needed, and also is preserved for the case of catalog being redeployed.

"Importing in a small numbers" is also a bit not stable, and how small it should be that it can be done in the reasonable number of attempts?
Unfortunately, the inconsistency also destroys the confidence that all images have been imported.
And still - no reply from Adobe... and there is no any "formal" way to raise an issue ticket and get a formal reply.
or I am missing something?

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

Some of the following may have already been asked, but

Images are on the NAS with fast dedicated network.

 

Your images are on your NAS. When in LrC, you attempt to import, are you selecting ADD? Not Copy or Copy as DNG?

 

Recommend Add.

 

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

Images are on the NAS with fast dedicated network.

In your LrC catalog settings, have you chosen to create the xmp sidecar files? That would slow things down. That would write to the folder the photos are in.

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

And the above leads to

 

Looking on the disk’s activity - nothing happens on the disc/network side.

 

Are you referring to a folder on your NAS that contains the photos?

or

Whatever hard drive, whatever folder, your catalog is in?

 

Whatever drive, whatever folder you keep your photos on, as long as it is not the exact same folder as the catalog is in. If you do not create those xmp files. Nothing will be written.

 

But, where the catalog is, the catalog gets larger, previews are created, so, yes, folder space used gors up.

 

During import that is, Export a different can of worms.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

A ohh by the way

 

Face detection and Address lookup paused, correct?

 

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

@GoldingD ,It looks like I would need to add further details – happy to do so.
Face detection - off

 

Disk activity - the NAS is attached as a local drive - hence I can see the activities using the resource monitor (or even a process monitor - it just generates huge log, hence used in "emergency" cases).
I always use Add when import from the local drive and copy when from the card.
If copy from the SD Card - the destination is always in the "import" folder. Usually files are moved to the "final" decapitation after being processed.

 

.XMP files are generated when the file is touched in the Lightroom. Hence should not be an issue.

 

When I refer to the disk activities I mean: Light room has e.g. 80% of import completion on the top progress bar, and system stops showing the progress. If lightroom is not touched – the progress bar frizzes on one position (it might be different one depending on the process) and that’s basically it. Lightroom neither finishes the import nor shows any progress, nor does any real work, however it consumes some CPU “power”. In some situations, Lightroom might keep memory consumption increasing up to the limit then system starts swapping.
The only possibility in such cases is to stop import by pressing the red cross next to the progress bar.

 

and no DNG

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

The portion could be 1000, or it can be 15.000 images at once

During import, what settings for preview creation? With that many, just to speed import up, would think that not creating any previews would help. The can be created later. 

During import, any Develop presets being allowed? Would slow things down.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

Preview can be on or off - no impact on the issue.
No development presets
add the tagging during the import - on or off - also shows no impact on the situation

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

this makes me curious, not nessacarily helpful, but...

Images are on the NAS with fast dedicated network

So, what is your LAN infrastructure?

 

  • Computer LAN connectivity. Through Ethernet? Using a NIC? As in not WiFi.
  • NIC in computer?  What bandwidth/speed? 100 Mbps, 1,000 Mbps and 10 Gbps
  • Your NAS? What bandwidth/speed? 100 Mbps, 1,000 Mbps and 10 Gbps
  • Your NAS, if 10 Gbps, via one Ethernet cable? as in via a 10 Gbps port? Getting at, are you getting full capability.
  • Ethernet cables from computer to switch, from NAS to switch, Category? cat 5, cat 5e, 6, 7? Fiber?
  • Switch bandwidth/speed
  • Computer and NAS same switch? If not, capabilities of switches.

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

1Gb LAN, connection via the same / single switch, Cat 6 or higher cables are used.
NAS and the computer have NIC on the MB  - not extra components, almost a direct connection.
No Fiberoptic.

tested: NAS can coop up to 2,5 Gb LAN with no issues (tested via 10Gb connection using Intel 10Gb NICs) more was not produced / tested due to the limitation on "another side", not the NAS.
Switch capacity:

Fabric: 56 Gbps line-rate

Packet Forwarding Rate (64 byte): 41.664Mfps

Jumbo Frames (bytes): up to 10240

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

Disk activity - the NAS is attached as a local drive

 

So you have a NAS that you can treat as a hard drive and connect via a USB cable?

A NAS, not a external RAID Drive enclosure .

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

No USB

NAS share can be mapped using standard Windows network capablities

NAS supports SMB protocol

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

(tested via 10Gb connection using Intel 10Gb NICs) more was not produced / tested due to the limitation on "another side", not the NAS.

Most likely component causing that is your switch. Very few of us have purchased 10Gbe switches.

 

Also, not all 10Gbe switches provide 10Gbe on all ports.

 

Second component would be the NAS, Only newer NAS  for home use and NAS for big business have 10Gbe NIC's included.

 

I would not consider a 1Gb LAN as fast if communicating to a NAS is involved. Would expect long wait times, but would not expect complete failure.

 

Is that NAS timing out? Going to sleep?

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

It is an enterprise grade NAS, the same is true for the Switch and other parts of the equipment - in this area I am completely confident as I have a computer and networks engineer degree and have technological background and profession.
The NAS and network parts have been fully tested and there are no issues in that specific area, as this equipment is used for some other purposes as well.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

One thing apparently lost in translation. I think the one time an Adobe employee interacted above, that employee asked about what if you imported from an actual hard drive instead of NAS. The inquiry appears to have been incomplete or misdirected. I do not think the Adobe employee made a good statement.

 

So, I will ask separately, with a distinct idea in mind. Communication capabilities.

 

Inquiry.

 

What if you copy all those thousands of photos to an actual hard drive, it can be an external drive, although, if internal, just a bit better as to eliminate another variable (first variable LAN speed ). On that hard drive, create a folder structure (folder name, subfolder names) exactly like that on NAS (could just copy the entire folder). In LrC, import those photos.

 

Later you can redefine the folder location in LrC, should you want to.

 

P.S. where do you keep backups of your photos? Not the catalog, but the photos. I assume you want to use the NAS for keeping originals on as to free up actual local hard drive space, but where are the backups? Personally, I would do it the other way, original photos on local hard drive, backup on NAS (and another ext hard drive that I lock away)

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Jul 22, 2021 Jul 22, 2021

Copy link to clipboard

Copied

Back-Up - on another system
Unfortunately, the disk space on the machine where the Lightroom is installed does not allow to have thousands of files.
It has more than enough space for the Lightroom catalog, but ... just small fraction of the images catalog.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 22, 2021 Jul 22, 2021

Copy link to clipboard

Copied

Add another drive. If this is is a laptop/notebook, add an external drive. Placing your catalog on your fastest drive has advantages, also your Camera RAW CACHE,  Photos do not need to be on a fast drive, a nice external drive will do.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Aug 16, 2021 Aug 16, 2021

Copy link to clipboard

Copied

"Add another drive."

I need to have redundancy that data is not lost in case of HDD failure.

Unfoirtunately my laptop does not give me this opiton.

 

"a nice external drive will do"

ultimately the NAS is an "external" drive, also for the system.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 21, 2021 Jul 21, 2021

Copy link to clipboard

Copied

Was scratching my head as to why I was interacting with a several months old posting, relooked, and noticed because you had made a newer reply. And relooking at that, I see s good question has not been addressed.

 

 

 

 

And still - no reply from Adobe... and there is no any "formal" way to raise an issue ticket and get a formal reply.
or I am missing something?

 

 

 

 

That one response by what appears to be an actual Adobe employee is, or used to be (used to be defined as a week or two ago) a rare occasion. The Adobe Techs and/or employees rarely interact in these community pages. When they do (or did, things might be improving) they typically state, were sorry, and inquire as to Lr version, OS, and have you tried resetting the preference file, never to be here'd from again. Somewheres deeply hidden in the how to use these community pages is some sort of statement that the site supposedly for customers to interact, to communicate, (not for trouble tickets, etc) about clear as mud.

 

You might try the official feedback site where Adobe Techs respond.

 

https://feedback.photoshop.com/topics/lightroom-classic/5f5f2093785c1f1e6cc40872?cType=PROBLEM&page=...

 

 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Jul 22, 2021 Jul 22, 2021

Copy link to clipboard

Copied

At Author, Quote “ 

In a nutshell, I am importing a relatively big number of photos (150k+) into the catalog. It is a fresh import.

Res. This is an unusually high number of images for one import. What file type?

 

The behavior is the same every time I start the import:

Import starts, it progresses to some point and then no further visible progress.

Looking on the disk’s activity - nothing happens on the disc/network side.

Catalog seats on separated nmve SSD with sufficient disk space (400GB+ free, system SSD has 300GB+)

Res. Lightroom stores the Catalog file in a specific folder with several ancillary file the major one being Preview files which are required for it to function. I have 41,600 files in my Catalog the majority being RAW camera images and the folder is now 109GB. I have built standard previews, no 1x1 previews or Smart-previews so if I use my experience as a guide the folder that houses the Catalog will grow to about 400GB.

You, state the drive is 400+GB what makes you believe the space on that drive is sufficient?

 

Regards, Denis: iMac 27” mid-2015, macOS 11.7.10 Big Sur; 2TB SSD, 24 GB Ram, GPU 2 GB; LrC 12.5, Lr 6.5, PS 24.7,; ACR 15.5,; Camera OM-D E-M1

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines