Camera Raw GPU out-of-memory error during masking – RTX PRO 6000 Blackwell 96 GB – imageMaskTex E_OUTOFMEMORY

Description of Issue

Adobe Camera Raw repeatedly encounters a GPU out-of-memory error during normal GPU-accelerated image editing, particularly when working with masks.

The error displayed is:

“The graphics processor has encountered an out-of-memory error. Due to a run-time error, GPU acceleration has been reverted to Limited for the remainder of the session.”

This occurs on a very high-end workstation with an NVIDIA RTX PRO 6000 Blackwell Workstation Edition with approximately 96 GB of dedicated VRAM.

Importantly, the Camera Raw diagnostic log shows that GPU initialization completes successfully and that approximately 100 GB of GPU memory budget is detected. The actual failure occurs later when Camera Raw attempts to allocate/render an image mask texture.

The relevant Camera Raw log entries are:

Determined GPU3 hard status: success

*** GL Error: E_OUTOFMEMORY (hr=0x8007000e): imageMaskTex ***

*** GL Error: imageMaskTex ***

*** Error: dng_exception (200025) in cr_gpu_tile_render_task ***

TrimFootprint: 4683.3 MB purged from pools.

This is followed by repeated:

Unexpected re-entrancy in RedrawMonitor -- skipping

System configuration

Adobe Camera Raw: 18.6.0.2698

Adobe Photoshop: 27.10.0

OS: Windows 11, build 10.0.26100.9278

CPU: Intel Core i9-14900K

System RAM: 192 GB

GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition

Dedicated GPU memory: approximately 96 GB

NVIDIA driver: RTX Enterprise 597.06

Driver reported by Camera Raw: 32.0.15.9706

Camera Raw detects:

DedicatedVideoMemory = 101279858688

MemoryBudget = 100474552320

Only one GPU is exposed to Camera Raw. Camera Raw reports the RTX PRO 6000 as supported and GPU initialization/self-tests complete successfully.

Steps to reproduce

1. Start Photoshop.
2. Open a RAW image in Camera Raw.
3. Ensure Use GPU for image processing is enabled.
4. Perform normal image editing, particularly using Camera Raw masking/brush operations.
5. Continue editing the mask.
6. Camera Raw eventually reports a GPU out-of-memory runtime error and changes GPU acceleration to Limited for the remainder of the session.

The problem is not limited to very large/high-resolution RAW files. I can reproduce it with smaller RAW images as well.

Expected result

Camera Raw should continue using GPU acceleration normally. A workstation with an RTX PRO 6000 Blackwell and approximately 96 GB dedicated VRAM should not encounter a GPU out-of-memory error during ordinary masking operations.

Actual result

Camera Raw reports GPU out-of-memory and falls back to Limited GPU acceleration. Mask editing then becomes extremely slow.

Disabling Use GPU for image processing prevents the error, but masking performance becomes unacceptably slow, so CPU-only processing is not a practical workaround.

Troubleshooting already performed

The following have been tested without resolving the problem:

• NVIDIA RTX Enterprise driver 597.06
• Previous NVIDIA Data Center driver 596.36
• Verified only one GPU is exposed to Camera Raw
• Camera Raw GPU support reset
• Camera Raw GPU configuration deleted/renamed and automatically rebuilt
• Camera Raw cache increased from 40 GB to 80 GB
• Windows page file changed from a manually limited 4–8 GB configuration to System managed
• Windows restarted after page-file change
• Hardware-accelerated GPU scheduling (HAGS) is Off
• Tested with a single monitor
• Tested smaller RAW files
• GPU Open/Save acceleration disabled
• DisableGPUInference.txt workaround tested – no improvement
• Preview AI Settings On Rollover disabled – performance becomes extremely slow and this is not a practical solution
• GPU image processing disabled – prevents the error but results in extremely poor masking performance

Windows Event Viewer was also checked immediately after reproducing the problem. There was no corresponding NVIDIA nvlddmkm, Display, DxgKrnl, WHEA, TDR, GPU reset or display-driver crash event.

A Kernel-PnP Event 219 observed separately was traced to a USB/HID WUDFRd device and is unrelated to the NVIDIA GPU.

Important diagnostic observation

This does not appear to be simple exhaustion of physical VRAM.

Camera Raw detects approximately 96 GB of dedicated VRAM and approximately 100 GB of GPU memory budget, initializes the GPU successfully, and then specifically fails at:

E_OUTOFMEMORY (0x8007000e): imageMaskTex

followed by:

dng_exception (200025) in cr_gpu_tile_render_task

The failure therefore appears to occur in Camera Raw's GPU mask texture/resource allocation or rendering path.

I can provide the complete Camera Raw Log, Photoshop System Info and DxDiag information if required.