What type of hardware do I need to export 25-30 K photos in 12 hours?

[john-arildf90553973]

Good evening,

Currently we have a record of exporting just under 60K photos in 12 hours with 4 pcs.
Each of these pcs has a limittation of around 18K photos with their current configuration.
So, one day one of my guys tried to export 23K photos from one machine and it was not done
exporting it when we came the next morning, so we had to wait about half of that day too for
the machine to finish the 23K photos. And with these pcs the CPU usage is at 100% all the time,
so the machine sadly cannot be used for other things.

So I am now looking for advice for a pc hardware setup that will allow us to export 25-30 K in 12 hours.
And I am willing to spend good dollars to achieve it.
I am aware that this will depend on the file sizes etc.,
so lets us just say it could be anywhere between 10-20 MB per photo.

Using (Link Removed by Moderator) I have come up with the following hard ware suggestions:

Userbenchmark PC Build Comparison
Baseline Bench: Game 203%, Desk 93%, Work 313%

CPU: AMD Ryzen TR 3960X $1,350
GPU: Nvidia RTX 3090 $1,500
SSD: Samsung 970 Pro NVMe PCIe M.2 1TB $300
RAM: G.SKILL Trident Z DDR4 3200 C14 4x16GB $639
Total: $3,789

I read the AMD Ryzen TR 3960X processor was better than Intel or otherAMD processors if you wanted to export huge amounts of photos (Link Removed by Moderator).

Anyone else have experience and suggestions for this type of huge amont of photo exports are welcome to reply.
Thanks for your input!

John

[ManiacJoe]

As you have pointed out, Puget Systems is the shop I would be looking to for recommendations.

Just out of curiosity, what is it you are doing that requires that level of throughput?

[john-arildf90553973]

Hello Joe,

We do School Photography

[Just Shoot Me]

And the Schools you take photos for has 25,000 > 30,000 students and you do all of them in One Day?

[dj_paige]

You want a much faster CPU to do the export faster. That specific CPU ought to do fine. Improving GPU, memory, disk speed will not affect export speed other than by trivial amounts.

I don't trust the recommendations from Puget Systems. Their job is to sell computers and make a profit. They want to sell you high end components that maybe you don't need. They make incorrect statements about speeding up Lightroom Classic on their web site. If they are telling you that you need 64GB of RAM to do this, then I consider that to be a good example of selling you stuff you don't need, as Lightroom Classic speed is not normally improved by RAM over 16GB.

[john-arildf90553973]

Well, We do have an i7-processor now and 16 GB RAM and like I said the CPU is at 100% all the time so we cannot use the PC for other things while exporting really.

[Todd Shaner]

In addition to the components you have listed using a separate NVME PCIe or USB 3.1 SSD drive for your Export Destitnation drive will help to increase speed.

[john-arildf90553973]

Do you reccomend the export to be on another drive than where LR is located?
Say,  SSD disk 1 with Windows 10 and LR and SSD disk 2 is where the export is output to?

[Todd Shaner]

Yes, that will reduce the export file write time and latency, which should speedup the export. It can be a local NVME PCIe SSD drive or a fast USB 3.1 SSD portable drive. Try it with one of your current systems.

[dj_paige]

I have to disagree. Disk speed is a trivial component of Export time. The write speed of any disk is much faster than the CPU can create files to write, and so the speed of the disk is not a bottleneck. Almost all of the export time is taken up by the CPU. You can run the system resource monitor during your export and see which components are being heavily used.

@john-arildf90553973If your i7 (unspecified, which i7?) isn't getting the job done in a reasonable amount of time right now, a faster CPU will get the job done faster. Not all i-7s are equal, some are faster than others, and there are i-9 available now as well.

[Todd Shaner]

"You can run the system resource monitor during your export and see which components are being heavily used."

Did that and I see no more than 7.0 MB/s when exporting files in LrC 10.0. Next I exported 20 Canon 5D MKII 21 mp raw files to full size JPEG with 80 Quality and destination drive my C: drive, which is a Samsung 970 EVO 1 TB SSD. That took 37 sec. (1.85 sec/file) I repeated the same operation with the destination drive a much slower WD MyBook USB 3.0 external drive, which took 33 sec. (1.65 sec/file), or about 10% faster.  This is most likely due to a reduction in disk queue latency. The SSD write transfer speed is ~2,000 MB/s and the WD MyBook drive is ~100 MB/s. If I had a 2nd SSD drive to test I'm guessing the difference would be even greater (15%-25%).

Anyhow it's easy enough to test.

[john-arildf90553973]

That is pretty cool. On one of our pcs we have 2 SSD discs, so we will test on that one,
but still I am thinking the main component might be the processor, then the graphics card,
then the SSD discs and then the RAM. Not sure if you agree?

[Todd Shaner]

As dj_paige mentions the LrC export process is very CPU intensive. Adobe has gotten better at utilizing multiple core processors so a higher performance processor 6 or 8 core processor will have the largest impact on reducing export time. Of course that's at a large cost by % of performance increase. If using an SSD as the destination drive can reduce the export time further (15%-25%) that's a pretty cost effective component. Let us know what you find out with your testing.

[Todd Shaner]

I forgot to add that you should make sure the 2nd SSD used as the destination drive does contain the image files being exported or for the camera cache, etc. In other words as a stand-alone drive, which is what we are trying to test.

This may be helpful for processor selection.

https://www.cpubenchmark.net/high_end_cpus.html

You can add multiple CPUs to the compare queue for detailed analysis (scale icon in upper right-hand corner).

[john-arildf90553973]

Finally in my office again.

The processor is an i7 7700 3,6 GHZ.
Yes, I am between a i9 processor and the AMD Ryzen TR 3960X.
The AMD Ryzen TR 3960X I read was recoomended for exporting huge amounts of photos
which is my case, but I do not know. Wish someone had experience to tell me their experiences on this.

[JP Hess]

Here's a radical idea. I don't know if it would work. Consider having a couple of high-end machines for overnight exporting, and, ahead of time on other computers, export these high-volume catalogs, splitting them. Then put one catalog on each of the two high-end computers to export the images overnight. Just a thought that came to me.