• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
10

P: Denoise AI and Pentax K3 MKIII DNG files - not working

Community Beginner ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

I'm not seeing any noise reduction using Denoise AI on Pentax K3iii DNG files in Lightroom Classic v12.3. Any future solution to this issue?

Thanks

Tom

Bug Investigating
TOPICS
macOS

Views

5.2K

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines

correct answers 1 Pinned Reply

Adobe Employee , May 04, 2023 May 04, 2023

I've opened a bug for investigation. I've downloaded two of your sample files to attach to the bug. 

Thank you for the report.

Note:  a change is required by the manufacturer before this issue can be resolved. 

Status Investigating

Votes

Translate

Translate
68 Comments
Adobe Employee ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

Would you be able to supply us with a sample file that does not exhibit noise reduction?

Rikk Flohr - Customer Advocacy: Adobe Photography Products

Votes

Translate

Translate

Report

Report
Community Beginner ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

Hello Rikk. Thanks for your response. Just took this high iso DNG with my Pentax K3iii. I imported into Lightroom Classic v 12.3 and saw no difference in the Denoise preview window and no change to the resulting enhanced DNG. I can't seem to upload the DNG. It keeps getting rejected with an error. I responed to you email with the attachment if that works.

Thank you.

Tom

Votes

Translate

Translate

Report

Report
Community Beginner ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

Having a hard time sending due to size. Sent email with link to iCloud for DNG.

 

Tom

Votes

Translate

Translate

Report

Report
LEGEND ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

Upload the raw to something like Dropbox.

Author “Color Management for Photographers" & "Photoshop CC Color Management/pluralsight"

Votes

Translate

Translate

Report

Report
Community Beginner ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

Votes

Translate

Translate

Report

Report
LEGEND ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

I got the file from Dropbox. 

I'm not sure what to make of what I see. The difference between Denoise on and off is really small, but there is a difference. I see it cleaning up the moiré, so it is doing something. Is it doing enough, or what it should? I don't know. 

I put the two screen captures into a folder on my Dropbox so you can view them in Photoshop up close. 

https://www.dropbox.com/sh/zauhspigiqj5hxs/AAApoPfQ_eYThPzzjbUTer31a?dl=0

Can you run one more test where you crank up ISO (or just under-expose what you shot a few stops, which is producing the noise)? 

Screenshot 2023-05-03 at 6.16.06 PM.pngScreenshot 2023-05-03 at 6.16.01 PM.png

Author “Color Management for Photographers" & "Photoshop CC Color Management/pluralsight"

Votes

Translate

Translate

Report

Report
LEGEND ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

[This post contains formatting and embedded images that don't appear in email. View the post in your Web browser.]

 

I downloaded a sample Pentax K-3 Mark III ISO 32000 photo from dpreview.com (not dead yet):

https://www.dpreview.com/sample-galleries/4949897610/pentax-k-3-mark-iii-sample-gallery/4691956898

 

Denoise shows considerable improvement on it:

johnrellis_0-1683161786652.png

johnrellis_1-1683161800863.png

 

Votes

Translate

Translate

Report

Report
Community Expert ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

Looked at the file and yes indeed. It appears Lightroom is only doing the raw details step (why the target cleans up in the moiré department) and not doing anything about the noise at all. For similar files from my cameras it will easily clean up the noise completely so it appears something is tripping up the noise reduction part.

Votes

Translate

Translate

Report

Report
LEGEND ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

"Looked at the file and yes indeed. It appears Lightroom is only doing the raw details step (why the target cleans up in the moiré department) and not doing anything about the noise at all."

 

The metadata of the denoised DNG indicates that LR thinks it applied Denoise:

 

 

$ exiftool -a -G High_ISO_TestShot_IMG8596-Enhanced-NR.dng | grep -i enhance
[File] File Name                       : High_ISO_TestShot_IMG8596-Enhanced-NR.dng
[EXIF] Subfile Type                    : Enhanced image data
[EXIF] Enhance Params                  : Adobe Denoise 1 Amount 50
[XMP]  Enhance Details Already Applied : True
[XMP]  Enhance Details Version         : 218300416
[XMP]  Enhance Denoise Already Applied : True
[XMP]  Enhance Denoise Version         : 251658240
[XMP]  Enhance Denoise Luma Amount     : 50

 

 

 

 

Votes

Translate

Translate

Report

Report
Community Expert ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

Interesting! Yet it clearly doesn't do it at all. Also tested the file in camera raw and it gives the same result of no noise reduction whatsoever. I also downloaded the image you linked to from dpreview and it does do a little bit of reduction there, but very badly and patchy. Far worse quality than what I get on my images (not from the same camera) and very strangely patchy and actually killing structure that obviously belongs in the image. Here for example is some woodgrain in the image @johnrellis linked to that gets pasted over by enhance (run in camera raw in this case but there is no difference with Lightroom). It have never see it do anything like this with images from my camera. In fact it often finds detail you can't even see easily in the original. Here it just pastes over the grain. Clearly it is not tuned correctly for this camera.

Screenshot 2023-05-03 at 7.33.29 PM.pngScreenshot 2023-05-03 at 7.33.06 PM.png

 

Votes

Translate

Translate

Report

Report
LEGEND ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

quote

Here for example is some woodgrain in the image @johnrellis linked to that gets pasted over by enhance (run in camera raw in this case but there is no difference with Lightroom). It have never see it do anything like this with images from my camera. In fact it often finds detail you can't even see easily in the original. Here it just pastes over the grain. Clearly it is not tuned correctly for this camera.

 

 


By @Jao vdL

Well, maybe "not tuned" for this camera (is that a thing?), but it is doing something, can we agree? Because the OP is saying: I'm not seeing any noise reduction using Denoise AI.

Seems it is (if not ideal). Does that sound right?  

Author “Color Management for Photographers" & "Photoshop CC Color Management/pluralsight"

Votes

Translate

Translate

Report

Report
Community Expert ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

That is true. There are areas where it doesn't do anything at all though. Also, in the OP's image I can't find any area where it did any significant noise reduction at all and I looked around all over. The noise has a bit smoother appearance and perhaps it has lost a bit of color noise but otherwise it is as heavy in the denoised image as in the original.

Votes

Translate

Translate

Report

Report
LEGEND ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

quote

That is true. There are areas where it doesn't do anything at all though. Also, in the OP's image I can't find any area where it did any significant noise reduction at all and I looked around all over. The noise has a bit smoother appearance and perhaps it has lost a bit of color noise but otherwise it is as heavy in the denoised image as in the original.


By @Jao vdL

There isn't much noise, to begin with (due in all part to the OP optimally exposing the image as shown in RawDigger). Hence, I suggested that the OP considerably under-exposing a test image, producing more noise, and trying again. 

Author “Color Management for Photographers" & "Photoshop CC Color Management/pluralsight"

Votes

Translate

Translate

Report

Report
LEGEND ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

quote

I downloaded a sample Pentax K-3 Mark III ISO 32000 photo from dpreview.com (not dead yet):

https://www.dpreview.com/sample-galleries/4949897610/pentax-k-3-mark-iii-sample-gallery/4691956898

 

Denoise shows considerable improvement on it:


By @johnrellis

And again (the ISO 32000 is moot), it IS super noisy because it is super under-exposed, which again is why I've asked the OP, who did such a great job exposing the example he provided, to do what the DP folks couldn't do.

Look at what is really happening to this K-3 raw data:

 

Under-Exposed and noisyUnder-Exposed and noisy

Now I'll get off my regular soap box about how high ISO has nothing to do with noise. 😜

Author “Color Management for Photographers" & "Photoshop CC Color Management/pluralsight"

Votes

Translate

Translate

Report

Report
Community Expert ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

? I see loads of noise all over, especially in the shadows. Enhance denoise should easily take care of this. Similar images from my Nikons will render completely clean after an enhance step.

Here for example. Look at the receptacle (zoom to 1:1 to see it). That is very very noisy. Enhance (left) didn't do anything to it. 

Screenshot 2023-05-03 at 8.09.42 PM.png

Votes

Translate

Translate

Report

Report
LEGEND ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

quote

? I see loads of noise all over, especially in the shadows.

 


By @Jao vdL

Let's be careful what we state is noise (that Denoise should deal with); the data from just the raw data shows a different data point:

Screenshot 2023-05-03 at 8.38.26 PM.png

The DP Review image is super noisy, and super under-exposed, and Denoise did deal with it, albeit not ideally in some areas. 

There is no editing settings applied above, unlike what you've shown us and shown at 125% zoom of that data. This is a mode (RGB composite only):

  • Raw Composite: for an RGB(G)-Raw it is an RGB-image where the green channel is

    computed by averaging two green channels available in the raw file. For all raw file where the CFA is not RGB(G) (but rather CFA is of a different type, like CMYG, RGBE, etc.) a grayscale image is displayed (i.e. Show RGBG in grayscale setting /see below/ is forced on for such files).

 

Denoise used on the OP's image isn't, not doing anything. It might not be correct, and I'm open to that possibility.

Denoise on the DP Review's noisy underexposed image from the same camera IS doing something.

So again I'll go back to my request: Can you (the OP) run one more test where you under-expose what you shot a few stops, and see if Denoise is reducing the noise? 

Author “Color Management for Photographers" & "Photoshop CC Color Management/pluralsight"

Votes

Translate

Translate

Report

Report
Community Expert ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

Both your screenshot from rawdigger and mine show extremely clearly that this is photon shot noise exactly like you expect from the exposure at ISO 6400 1/8s f6.7. Underexposure has nothing to do with this. A well exposed image at those settings will have tons of photon shot noise. It is just simple physics and it is plainly obvious from the images. Similar images from other cameras with exactly the same kind of noise are easily completely cleaned up by enhance's denoise (example below). It just does a terrible job with images from the OP's camera including the image from dpreview which is nowhere near as well cleaned up as images from other cameras with similar lighting. It is very clear to me that the denoise code is not dealing well with images from this camera for whatever reason.

Here is a very similarly ISO6400 correctly exposed image from one of my Nikons with a similar sort of in-house scene. Completely cleaned up by denoise exactly as advertised. This is what enhance denoise should be doing to the image from the OP but it isn't doing much at all. Enhance's denoise is extremely good when it works but it clearly isn't working right here.

Screenshot 2023-05-03 at 8.25.26 PM.png

Votes

Translate

Translate

Report

Report
LEGEND ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

What your Nikon shows is moot. Apples to Oranges. This isn't good science/testing. 

Underexposure has everything to do with producing noise. This is why one image is super noisy on the same camera model. And Denoise works. 

 

ISO doesn't produce noise; under-exposure produces noise. An ISO 100 image can be massively noisier than an ISO 800 from the same scene and camera. 

I don't know if your ISO 6400 1/8s f6.7 is ideally exposed since I can't view it (yet?) in RawDigger. 

This is really simple; the OP should underexpose some test images at any ISO, then it will be noisier than not doing so, and the OP can tell if he/she sees Denoise doing something. Just as we see with the DP Review capture using the same sensor. 

Let's concentrate first on the question asked: 

I'm not seeing any noise reduction using Denoise AI on Pentax K3iii DNG files in Lightroom Classic v12.3

I am seeing something; I think you and others have said they see something. I see it cleaning up the moiré, so it is doing something. Is it doing enough, or what it should be? I don't know. Yet. With more data and testing, not assuming this camera behaves like a Nikon or exposed (?) may get us closer to the answer. 

Here's what I know: the idea that shooting a high ISO should always produce noise; thus, Denoise should do something is fundamentally wrong thinking. Having a noisy image (due to underexposure) should be when users look at using Denoise. Which is why it is thankfully not called Denoise-highISO.  🤔

Author “Color Management for Photographers" & "Photoshop CC Color Management/pluralsight"

Votes

Translate

Translate

Report

Report
Community Expert ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

>What your Nikon shows is moot. Apples to Oranges. This isn't good science/testing. 

Underexposure has everything to do with producing noise. This is why one image is super noisy on the same camera model. And Denoise works. 

 

Yeah that is not correct. Underexposure DOES NOT PRODUCE NOISE. That is NOT how the physics works. Exposure combined with light intensity produces noise. Every image has noise as you can't avoid physics. On modern cameras which all these are, the signal is ISO invariant because the amplifiers have gotten so good that noise is caused only by the number of photons hitting the sensor pixel. ISO just scales the number. Doesn't matter whether the entire image is over or under exposed. Doesn't matter what camera. All that matters is the intensity hitting the sensor locally and what the sensitivity of the sensor pixel is. The noise is proportional to the square root of the total number of photons hitting the sensor pixel. This is just simple physics. This sort of noise is called Poisson noise or shot noise. ISO does not matter, under or overexposure does not matter. All the images I show are without any exposure bias and zero edits, so all you need to do is look at it and see the noise. The data is obvious. The same for the OP image. The noise has the exact same physical origin and is plainly obvious to see.

 

The Nikon image is precisely relevant because it shows what the algorithm should be able to do. I could show images from any other camera but the camera does and should not matter! These images were similarly exposed at a similar light intensity with a similar light sensitivity sensor. The physics is the same, and most importantly, the visual noise in the images is the same which is all that should matter to the AI algorithm. It is just not doing the expected noise reduction for the OP's camera and it is not doing it for the dpreview files where the noise reduction is really poor and nowhere near comparable to images from other cameras at similar conditions. The fact that the algorithm is not dealing with visually exactly the same kind of noise means very simply that this is a bug. The algorithm should be able to deal with this.

 

>so it is doing something. Is it doing enough, or what it should be? I don't know. Yet. With more data and testing, not assuming this camera behaves like a Nikon or exposed (?) may get us closer to the answer. 

 

The answer is plainly obvious. It is doing almost nothing where it should be doing far more. There is no reason for it not doing this. This kind of noise is exactly what the algorithm was designed to deal with, yet it is not doing it or extremely poorly. You don't need any more data as what is shown here is overwhelmingly obvious. It is not dealing with these files correctly. There is no reason why. Probably just a simple bug.

Votes

Translate

Translate

Report

Report
Community Beginner ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

Tom here, the OP - Thanks for the attention to this issue. What I'm not seeing is the fairly dramatic noise reduction and "resolution enhancement" that i have seen on other demonstrations with other (non-Pentax) DNG files on places like YouTube. If I use the manual noise reduction sliders in LR on this DNG, that seems to produce a better result than from the AI process. I have some bird photography taken with my K3iii where I can post to further determine if the Denoise AI is doing what I would expect it to do - denoise and enhance feather detail.

Votes

Translate

Translate

Report

Report
Community Beginner ,
May 03, 2023 May 03, 2023

Copy link to clipboard

Copied

Votes

Translate

Translate

Report

Report
Community Beginner ,
May 04, 2023 May 04, 2023

Copy link to clipboard

Copied

Tom here, the OP - Thanks for the attention to this issue. What I'm not seeing is the fairly dramatic noise reduction and "resolution enhancement" that I have seen on other demonstrations with other (non-Pentax) DNG files on places like YouTube. If I use the manual noise reduction sliders in LR on this DNG, that seems to produce a better result than from the AI process. I have some bird photography taken with my K3iii where I can post to further determine if the Denoise AI is doing what I would expect it to do - denoise and enhance feather detail.

Dropbox link for Pentax K3iii DNG files:

https://www.dropbox.com/s/tmp78s87hudoksn/2023-04-08_Birds-7907.DNG?dl=0

https://www.dropbox.com/s/98caujm0w36dltz/High_ISO_TestShot_IMG8596.dng?dl=0

 

Tom

 

 

Votes

Translate

Translate

Report

Report
LEGEND ,
May 04, 2023 May 04, 2023

Copy link to clipboard

Copied

All that matters is the intensity hitting the sensor locally and what the sensitivity of the sensor pixel is.

 

 The intensity IS exposure! The amount of photons from Aperture and Shutter striking the sensor.

I'm not nor ever said the “issue” here is or is not a bug! I asked for more data (an underexposed capture that like the DP Review capture is underexposed and noisy and does undergo NR).

Why does that Capture show more results than the OPs? It is both massively noisy and underexposed. That kind of capture from the same sensor would be helpful here, not comparing it to an utterly different camera sensor and then making assumptions from there.

 

"Underexposure DOES NOT PRODUCE NOISE".

Ok but:

Details:  Canon R6 Mark II.  ISO 200 (yes, I can set it lower with a custom setting, no need).  Meter: Minolta Flash Meter III incident reading.   IncidentMeterNormal.dng: Normal exposure is recommended exposure of the Minolta meter: 1/15th@F8 In-camera meter tells me that would be 1 stop over exposed it recommends 1/30th@F8.   -3stops.dng 125th@F8 or three stops below incident meter recommendations: UNDER EXPOSED.  All settings in Lightroom Classic’s Detail OFF.Details: Canon R6 Mark II. ISO 200 (yes, I can set it lower with a custom setting, no need). Meter: Minolta Flash Meter III incident reading. IncidentMeterNormal.dng: Normal exposure is recommended exposure of the Minolta meter: 1/15th@F8 In-camera meter tells me that would be 1 stop over exposed it recommends 1/30th@F8. -3stops.dng 125th@F8 or three stops below incident meter recommendations: UNDER EXPOSED. All settings in Lightroom Classic’s Detail OFF.

 

Author “Color Management for Photographers" & "Photoshop CC Color Management/pluralsight"

Votes

Translate

Translate

Report

Report
Adobe Employee ,
May 04, 2023 May 04, 2023

Copy link to clipboard

Copied

I've opened a bug for investigation. I've downloaded two of your sample files to attach to the bug. 

Thank you for the report.

Note:  a change is required by the manufacturer before this issue can be resolved. 

Rikk Flohr - Customer Advocacy: Adobe Photography Products
Status Investigating

Votes

Translate

Translate

Report

Report
Adobe Employee ,
May 04, 2023 May 04, 2023

Copy link to clipboard

Copied

@thix2112  Can you confirm the behavior on "All eligible files", All Pentax KIII files or just some files?

Rikk Flohr - Customer Advocacy: Adobe Photography Products

Votes

Translate

Translate

Report

Report