P: Tif scans won't open after extensive Gen Ai removal work

[Patrick Baldwin]

I have some 16 bit scans that I have been working on in ACR 17.2. I have done very extensive retouching of the damage to the negative using the Generative Ai removal tool because this is easily the best result. File size is about 158MB. However it is incredibly slow in a way that it isn't when I do this on my DNG camera files. Now, returning to the file it won't open again in ACR, I just get a beachball. I left one "opening" overnight and it was open the next morning but I don't know how long it took to open. The file that has the very extensive spotting wouldn't build a preview unless left overnight and now will not open. 

I am bewildered as to what is going on. System is Macbook Pro M3 Max running Sequoia 15.3.1 with 36GB ram 1TB internal and accessing the files on an external NVME 4TB SSD 

[Jeff Arola]

To rule out the external drive as the cause, try copying the file to your main drive and see if that makes any difference.

[Patrick Baldwin]

I have the same problem trying to open them from my spinning external HD. I can open other files (DNG's) from the externall SSD as well. I am assuming it's something to do with the extensive retouching on one of the files but even files that have had no retouching but have been opened in ACR are problematic. Rescanning is absolutely a last resort. There are over 500 scans!

[Jeff Arola]

How big are the files in terms of size on disk?

For whatever reason external drives with macOS can slow to a crawl at times, so that's why i suggest copying one of the files to your internal ssd and then see if that copied file opens any faster in ACR.

[Patrick Baldwin]

They are 156MB. The external SSD has a read/write speed of 3600mbs. I am positive it's not a bottle neck. It is working absolutely fine on other files. I have one that has been trying to open for 2 1/2 hours at the moment.

I just tried your suggestion however but it still won't open.

Many thanks for responding.

[Patrick Baldwin]

The file eventually opened. It took 12 hours! Attached is a screen shot that shows the number of retouches using the generative Ai tool. I am fairly confident this is the cause of the problem. I assume it's because the file has to be able to present that retouch history so that I can undo any of it.

As I have finished the retouch I went to export it as another tif on the basis that that wouldn't have to "carry" the retouch history so would open easily. The problem is it is currently 10 minutes into the export and the progress bar went straight to 50% ish and has not changed in 10 minutes so now I have to wait until that exports assuming it will eventually!

[D Fosse]

This is most likely a corrupt file.

The main problem with external drives is all the extra layers of the interface, USB drivers, protocols, etc. That results in a vastly increased risk of file corruption - especially if you save directly to it. Never do that, save locally, then copy over.. Not only because of all these extra interface layers that complicate the save process, but because of worn or loose cables and connectors. It only takes a nanosecond broken connection, and your file is toast. Never touch the drive while transfer is in progress.

(the speed you mention is the nominal speed. You never get that in practice because of all the extra layers. An external drive will never be nearly as fast as an internal drive running directly over a PCIe bus)

[Patrick Baldwin]

Thanks for responding but I am going to disagree here. I have been saving to external drives for 28 years. I have never had a problem with dodgy files, unexpectedly slow speed etc. It isn't a corrupt file. If it was corrupt I don't think it would open at all ever.  All 528 are problematic and it is something to do with ACR and tif and generative Ai. That SSD speed is as tested with Blackmagics speed test. It may not be as fast as the M3 internal but it is plenty fast enough and is emphatically NOT the problem here. Please see the attached speed test screen shot which is a test on a 5GB file.

[Patrick Baldwin]

So the file saved which took something like 20 minutes. The export now opens instantly in Photoshop or in ACR so with that in mind I am confident this isn't a corrupt file otherwise I wouldn't be able to get this far.

[D Fosse]

@Patrick Baldwin 

It works until one day it doesn't.

https://helpx.adobe.com/photoshop/kb/networks-removable-media-photoshop.html?x-product=Helpx%2F1.0.0... 

"Technical Support strongly recommends working in Photoshop directly on the local hard disk. To prevent data loss, save files to your hard disk first. Then transfer them to the network or removable drive in the Finder or in Windows Explorer"

(my emphasis).

[Patrick Baldwin]

Well yes but after 28 years I feel confident in my workflow. Plus that article is weighted VERY heavily towards working over a network which is not something I would ever do knowing these pitfalls. Below is from the same article so I feel you are being selective here. The fact is as I have shown my problem is NOT a corrupt file otherwise I wouldn't have got as far as I did.

Important: External hard drives should work with Photoshop without a problem, although depending on how they are connected, might be slower than working with files on your internal drive. Testing against these drives by temporarily disabling them and working exclusively on an internal drive is appropriate. However, Adobe is not stating that there should be regular problems storing files and working with external hard disks. 

[D Fosse]

Well, you are having a problem opening files, right? So you can't say it works without issue.

What I'm saying is - take this out of the equation first. Resave the file to your internal drive, see if it opens OK now. Copy it over to the external, copy it back, check if it opens normally.

[Patrick Baldwin]

I have. Please read everything I wrote. It takes ages to open them from multiple hard drives. They do open, I can work on them, they take a long time to export. I have an exported tif that is fine and opens as I would expect. If the original was corrupted how exactly would I get this far? In my 28 years any corrupt file just doesn't open and gives an error message. This is not happening here. Please see the attached file. This is the final from the file I am talking about. How do I get this if the file is corrupt? Please explain how? I am not saying there isn't an issue but you seem to be ignoring all the evidence that shows it isn't a corrupt file.

Once the file opens, I can work on it and export a tif that opens perfectly. Corrupt files don't behave like that. See for yourself.

[D Fosse]

OK, calm down. So if the file isn't corrupt, there is still an obvious problem here, because those timings are not normal.

And what I'm saying is that saving/opening directly from an external drive is a known potential problem. This isn't the first. We get posts like this all the time. It is always the prime suspect - and usually turns out to be the actual problem - in cases like this.

Nobody ever said it always causes problems. The point is that sometimes it does, and that can happen unpredictably. That's why it's not recommended.

All that said, if these problematic TIFFs all come from a scanner, that is another unknown factor to look into.

[Patrick Baldwin]

You were telling me try things that I had done yesterday and had posted about. I don't know how to say this any more clearly but my external SSD is NOT problematic. You keep saying it's not recommended. Even Adobe clearly state as I quoted that they are NOT saying don't use external hard drives. In 28 years of doing this I have never met a photographer who didn't open and save from external hard drives. Every copying to and from hard drives that I do is run with software that verifies. I never use the finder which doesn't verify.

This issue arose after some processing in camera raw. The scanner is nothing to do with it. 

Please don't tell me to calm down. I find your fixation in the face of everything I have posted and all the evidence to the fact the file itself isn't corrupted to be exasperating. Camera raw caused this problem. Did you even read my last post where I attached a final tif from the file you say must be corrupted? I asked you to explain how but you haven't. For the last time: I know there is an issue. It is nothing whatsoever to do with my external hard drives. Please stop saying it is.

[Conrad_C]

The hardware is certainly powerful so the fault is probably not there. My M1-based MacBook Pro is much less powerful than yours, and it has no problem with scans of that file size. I don’t think 158MB is very large for a TIFF. I also edit off of external NVME SSDs, so there should be no problem there either. In other words, if I was having that problem I would be bewildered too.

I’m not sure what the hangup is with Camera Raw re-opening those TIFFs, but if I had that problem I might try opening and re-saving the TIFF file in other software such as Photoshop in case it writes it back out maybe differently enough to fix the problem. Yes, there is a risk that the correction metadata might be lost, so test that with a copy of the TIFF file. Sometimes I will also try re-saving a file through some of the free/open source photo apps.

Another option is to see if Lightroom Classic is any better at displaying that file, in case its code is different enough that it doesn’t run into the problem. 

I suspect the problem is somehow related to the edit metadata. Otherwise a TIFF file of that size should not pose any kind of a challenge on the hardware you’re using, where it should open quickly.

Is there a sample TIFF file that you would be willing to attach to this thread if you want to see if someone else can load it successfully? Because if no one else can, I would guess that someone at Adobe would want to look at that file in case there is a metadata-related performance bottleneck they didn’t anticipate.

Another perspective: The next time you try to load an affected TIFF file into Camera Raw, open macOS Activity Montor and watch the CPU and Memory tabs. Do either show a spike in resource usage when loading that file, like 450% CPU usage or does the Memory Pressure graph rise into the orange or red zone?

- - -

Now for an unsolicited opinion…

I sometimes do similar work (restoring scans of old damaged film). Although it’s very nice that Camera Raw/Lightroom Classic edits are nondestructive, a high number of retouching edits or masks in a single image can slow down the software because it’s always re-checking all of the edits in case it has to re-render any. I came to the conclusion that it was much more efficient to use the Remove/Heal features in Photoshop because once applied, they’re rendered to pixels. I do add those retouch edits to a separate layer in Photoshop so that the original is preserved. I’m under the impression that the Remove tool in Photoshop should be as good (maybe even better) than the one in Camera Raw, if its generative AI option is enabled.

This way, when I want to do a bulk tone/color correction session in Lightroom Classic (works the same as Camera Raw), the TIFFs don’t come with all of the performance overhead of many spot retouches, since those were rendered down to pixels in Photoshop. As a result Lightroom Classic/Camera Raw only have to track and re-render the tone/color/detail edits.

[Patrick Baldwin]

Many thanks Conrad for taking the time to respond.. I have just sent the file and an exported tif out of ACR to Adobe and they will report back I am sure.

I can't load an example because the limit here is 47MB and I have attached an A5 version of the exported file in an earlier post but hat doesn't help you in regards to testing. I will attach them here again with the screenshots of the retouch.

If I bypass ACR with the file it opens instantly in Photoshop but without any of the ACR retouch or processing. It's effectively the raw scan before any ACR work.

I will try a test with the remove tool in Photoshop. I find the original tools to often put a little softness/blur around the retouch spotting and the Ai tool doesn't do that. I was surprised how slow the Ai tool operated because on my normal diet of 50mb dng files it just whistles through with me brushing as fast as I like. I do need to find a faster way that is as good if I can because doing 529 like this is never going to happen.

[Rikk Flohr: Photography]

Crossreferencing to internal bug ticket.