P: Zooming very large images shows grey box

[Crystal Visionary]

I am using lightroom Classic CC 7.0.1 and Camera Raw 10.0. Is there a limit to how large a file can be and still be able to create a 1:1 preview?  I have a 16 bit TIFF that is 26246 x 2899 pixels.  When I try to view it at 1:1 or any other size except "Fit" in the Library module, it refuses to draw (Fill works so long as I don't try 1:1 first).  I just see a grey box. I can view it in any size in the Develop module or Photoshop, but not the Library module.  Does the Library module have a size limit, and if so, what is it?  I also checked the Task Manager and it showed the CPU and disk had little or no activity, and memory was using 15GB which is not unusual on my system. I have 3.67 Terabytes of unused disk space. I have an i7 6700K processor running at 4.00GHz and 32GB of DDR4 Sdram memory and Windows 10 version 1703.

[Todd Shaner]

I'm seeing the same behavior using LR Classic on Windows 7. See this post: https://forums.adobe.com/message/9956218#9956218

[johnrellis]

I observe the same behavior with a 20K x 13K PSD (260 megapixels) on LR CC 7.0.1 / Windows 10 running in a virtual machine, but only if I repeatedly and quickly click between Fit, Fill, 1:1, and 1:8.

On MacOS 10.12.6, I don't see the grey box, but I do get a huge virtual-memory leak after a minute or so of such clicking in Library Loupe:



Do you see a similar increase in virtual memory in Windows Task Manager? It may be that an allocation of virtual memory is failing on Windows, causing the grey box.

[Todd Shaner]

John, I don't think it has anything to do with memory. A 10000x10000 (100 megapixels) file and 16000x2000 (32 Megapixels) file render at 1:1 Zoom view with  no issues, but a 16500x2000 (33 megapixels) file shows nothing in the Loupe with 9.1 GB memory available. Weird!

[Crystal Visionary]

I have trouble the first time I try it.  I  have tried closing Lightroom and then reopening it and still had the problem right away.

[Crystal Visionary]

When I try to render the image at 1:1, I see no increase in CPU, Memory, or disk activity.  Virtual memory is set at 101 GB on Disk D, but with 32GB or RAM. I have never seen any evidence of paging or the use of Disk D.  I have added a screen capture to show you what I see.

[Crystal Visionary]

The file is 435 MB in size.

[johnrellis]

" I don't think it has anything to do with memory."

It may have something to do with virtual memory.  As I described on Mac, I can pretty quickly get LR to allocate 62 GB of virtual memory before MacOS warns me it's run out.   LR isn't actually reading or writing this memory, since on a machine with 16 GB of physical memory, that would take a very long time, with lots of paging to and from disk.

My hypothesis: LR is incorrectly requesting that the operating system reserve very large blocks of virtual memory, but LR isn't actually using that virtual memory (not writing into it or reading from it). Maybe LR is getting an error from Windows when it requests more virtual memory and shows the grey box as a result.

It's been years since I read about Windows virtual memory, but I (very) vaguely recall that Task Manager may not show such reserved virtual memory until it is actually touched by the program.

Here's what the memory situation looks like on Mac:

[Todd Shaner]

Virtual memory is showing at ~0.5% usage with 12GB system memory and the Windows 7 Page File set to 12.5GB.

DirectX support was added in LR Classic and causing many different screen rendering issues. Perhaps this is another one caused by DirectX incompatibility. I used the below procedure (#2) to force LR Classic to use OpenGL and not DirectX, but the blank preview screen is still present with the 16500x2000 image file at 1:1 Zoom view with Use Graphics Processor checked or uncheked.

https://helpx.adobe.com/lightroom/kb/crash-gpu-directx-enabled.html

Do you see the same 1:1 rendering issue using a 16500x2000 image file?

[johnrellis]

This morning I see the rendering issue occur pretty quickly on Windows 10 with both a 16.5K x 2K PSD and a 20K x 13K PSD.  I don't observe Task Manager showing virtual memory increasing unboundedly like it does on Mac.  

However, it may be that on Windows, the hypothesized incorrect VM request from LR is being immediately rejected by Windows with an error, causing the grey screen -- if this is the case, then it definitely won't show up in Task Manager.  Or it could be that WIndows is allocated the VM but that Task Manager isn't reporting this allocated but untouched VM.

Or it could be that the VM leak on Mac is unrelated to the rendering issue on Windows.

[Todd Shaner]

Regardless of what's causing the 1:1 Preview rendering issue in LR Classic I'm not seeing it in LR CC 2015.12. It displays the 1:1 Preview for all of these file dimensions including the maximum combined limits of 65000x8259 (536.835 Megapixels).

A file with dimensions of 65000x8260 or 65001x8259 will NOT open in LR CC 2015.12 or Classic so the actual limits appear to be 65,000 Pixels/Edge or 536.870912 Megapixels (2_29th Power).

[johnrellis]

"the actual limits appear to be 65,000 Pixels/Edge or 536.870912 Megapixels (2_29th Power)"

Adobe's documentation says "Files with dimensions greater than 65,000 pixels per side or larger than 512 megapixels.".  I think they were thinking of "mega" defined to be 2^20, but as far as I can tell, nearly everyone else in digital photography defines it as 10^6.  The definition of 2^20 is still used by some (including Windows) when referring to bytes.  See here for more nerdliness: https://en.wikipedia.org/wiki/Megabyte

[Todd Shaner]

The 65,000 pixel per side limit appears to be an arbitrary decimal value and the 512 Megapixel limit the binary value 2^29 (536.870912 Megapixels).

Any size image file with a side >65,000 pixels or >536.870912 Megapixels will not Import into LR.

[Crystal Visionary]

My image is much smaller than the stated limit, but will not draw in the Library module, but it will in the Develop module.  My machine does not show any use of the virtual memory.  One interesting possible explanation may be that it has something to do with the graphics card.  Adobe documentation states that only the Develop module uses the graphics card for GPU acceleration.  So, there is a difference between how the library module renders images and how the Develop module does it, and how the GPU is used.

[Todd Shaner]

That's true, but if 'Use Graphics Processor' in LR Preferences is unchecked the Library and Develop modules should use the same rendering and display path. Unfortunately doing that doesn't "fix" this 1:1 Preview rendering issue. Adobe added DirectX support in LR Classic, which may be the root cause of this issue. I found a similar issue at the below post, which is now marked 'In Progress' so Adobe is aware of the issue and working on a solution.

https://feedback.photoshop.com/photoshop_family/topics/lr-classic-noise-reduction-effect-not-display...

[Dwight McCann]

Lightroom CC Classic 7.1 [1148620], Win 10 Pro, Nvidia GTX 1050 [23.21.13.9077]. With GPU enabled, some very large images will not display 1:1 in the main window. They will display 1:1 in window 2 Loupe. They will display in Develop module. Turning off GPU acceleration resolves the problem. I have tried switching color monitor profiles to sRGB without effect.

[Dwight McCann]

The issue in the noise reduction thread is apparently fixed but the problem I report below has not been fixed.

[Todd Shaner]

The noise reduction issue in that thread has NOT been fixed and is still marked as 'In Progress.' So if related to your issue then your issue is not fixed either.

[Dwight McCann]

Interesting ... I thought a couple of posters over in Family said it was resolved.  But you are certainly correct that my issue has not been resolved ... but now that I realize that it is not an issue that Adobe care about I'm just going to turn off GPU until I get another update.

[Todd Shaner]

I think they meant it was resolved when they unchecked Use GPU, which was one of the suggestions. Adobe is working on a fix, which will hopefully be in the next update.

[Dwight McCann]

Only time will tell! As a 35+ year programmer on mainframes and mini's for the University of California I believe two factors influence what code is added/modified: the programmers want to work on new exciting features and could care less about program errors which has a workaround; and pressure from Sales for new features with not a single thought given to the errors they have already sold! Without solid competition, Adobe has no motivation to allocate resources to address coding changes that don't result in revenue.
 

[Crystal Visionary]

That is quite evident on the database management side.  Lightroom's abilities as a database manager are mediocre at best and I can't think of any improvements they have made since Lightroom 1.  They seem to view it as a RAW processor with just enough Database management ability to get by.

[Dwight McCann]

Yup, they really need an 'Advanced' facility for the database.  All the tools are there in the code support, they just need a frontend.

[FeedbackCommunityMember]

For what it's worth, I've just tried disabling GPU acceleration and it didn't solve my problem. Running Classic CC release 7.5 with Camera RAW 10.5.

[Todd Shaner]

Since this issue hasn't been addressed (i.e. fixed) by Adobe I started looking at alternatives.

It appears the LR Preview is limited to ~16,384 pixels (2 to the 14th power) long Edge dimension with 16,385 failing. I have numerous LR Panorama DNG files that are longer than this dimension and I certainly don't need that much resolution for most of my usage. If you're shooting panoramas with a 50+ Mp camera it's very unlikely you will need the full-resolution panorama merge DNG file.  I happened to fall on an old post discussing how to reduce the size of these large DNG files. https://forums.adobe.com/message/10254121#10254121

Use the below LR Export module settings to covert large panorama DNG files into resized lossy DNG files that are 1:1 view enabled. Using even a very wide 1:5 aspect ratio panorama the dimensions will be 3250 x 61250 (52 Mp), which is capable of creating a 300dpi 10.8" x 54" print. I converted a large 17844 x 3806 panorama using the below settings to 16250 x 3466. Next I exported the original DNG file to TIFF 16 bit ProPhoto RGB format with Resize to 16250 Long Edge and Screen Low Sharpening. At 1:1 or even 2:1 view there is no visible difference in the image quality between the loss-less TIFF export file and the resized lossy DNG file. Even the fine gradient sky areas are identical with no banding observed.

Below is a comparison of file sizes. The Resized Lossy DNG file is slightly smaller than a same size 80 Quality JPEG Export file. I'm not sure what type of compression is used for the lossy DNG file, but they also look identical.

Having said all this it still begs the question why Adobe hasn't fixed this issue since it was working in earlier versions (pre LR Classic).

[Todd Shaner]

Thinking about this some more I wondered if the reduced-size Lossy DNG could be used to sync editing settings back to the original full-size LR panorama DNG.

To test this I used the original 17523 x 3906 panorama DNG file to create a lossy DNG file with Long Edge 7856. This is less than 1/2 the original resolution to see if resizing affects the Develop setting sync. I applied EVERY Global and Local setting available in LR (except Red Eye Correction) to the 7856 x 1706 Lossy DNG and then synced it to the original DNG file with no settings applied. The results are identical when viewed at any zoom size.

This allows using the much smaller file size lossy DNG as a settings archive. It also acts as a Smart Preview if you set 'Resize to Fit' at 10000 px or less when creating the lossy DNG. This will improve editing performance without having to enable the global Preferences setting 'Use Smart Previews instead of Originals for image editing.' Make sure to stack the original raw files with the lossy DNG copy, which I always do anyhow. If at a later date you need a full-size copy of the panorama rebuild it using the stacked raw files and then Sync All of the settings from the lossy DNG to the resurrected full-size DNG file. Done!

I hope someone finds this useful. Thank you.