Green noise in deep shadows.

[Imagewerx]

I'm using Lightroom Classic CC to edit Panasonic RW2 RAW files and always do my best to ETTR (Expose To The Right).But on a very few occasions my camera still under exposes and when I boost the shadows I'm left with very obvious green noise.I know I can (and do) remove it manually with the adjustment brush.Is there a way to do it automatically without having to use the brush for the rare times I need it now?

[TheDigitalDog]

https://forums.adobe.com/people/Todd+Shaner  wrote

LCD monitor test images

On my calibrated monitor I can see Black Level 1 and White Saturation level 254.

This is a good test too:

http://www.aardenburg-imaging.com/monitorchecker/

[Imagewerx]

We've had this same conversation on at least one occasion before this.

I'm an amateur photographer and take photos purely for my own enjoyment.I don't sell prints or have my photos hung in galleries for others to look at.

My monitor (a 55inch high quality TV set) is not 'calibrated' to a worldwide agreed laboratory standard,but it is calibrated to what my eyes (and brain) agree looks right.I'm the only person in the world that can see through my eyes and therefore am the only person who can judge what's right or wrong for ME.If I have to have the brightness turned up higher than most people find comfortable,then that is how it is and this can't be said to be 'wrong'.My eyes are far from perfect so for me this is how I need to do it.

On the above monitor test images on the top left I can JUST about make out number one (it is very faint),while bottom right I see pure white.On the white ones I can just about see the checkerboard on 244,anything above that is pure white.

It is also 'calibrated' against other things I view on it that I would expect to be correct,such as photos taken by people a lot better than me at this on photo sharing sites like Flickr.

Even turning the brightness down on my monitor,the white Mini in the photos above still looks to have the highlights blown out to me while the rest of the image is now under exposed.

I should also explain that the reason I under exposed was a throwback to my Canon days when shooting JPEGs I had to dial in -1 EV to get it consistently not over expose.I'm still getting used to the greater dynamic range that processing RAW images gives me,but fully accept the compromises in low light performance that a micro 4/3 sensor gives me.

[TheDigitalDog]

There is no worldwide agreed laboratory standard!

[D Fosse]

I'm not quite sure where to begin here.

First of all, a TV set is not intended to view or judge photographs. Accuracy is not even on the list of priorities. High quality TV does not mean high quality photo reproduction! Unless you get a decent desktop monitor, or at least a top-level laptop screen, you can't make any judgements. You simply can't. The fact that everything above 244 appears blown-out white to you, is a clear indication that your TV is miles off the map. For one thing, it clearly does not have a standard tone response curve.

No, there is no worldwide agreed standard as such. But there is a way to bring us all on the same page - a common reference that we all share: a printed photograph on a piece of paper. That's what you want to see on screen. Monitor white should be a close visual match to paper white. Once you have that match, the rest will more or less fall into place by itself.

The obvious problem with that is that it's a moving target, entirely dependent on the environment. Print viewing light is one thing, but ambient light also influences your perception of the screen image. But this doesn't have to be scientific to be effective. Put your photo in what you consider to be adequate/good lighting, and adjust the monitor to match. Use your eyes. Take nothing for granted - you may end up with settings very different from what you expected.

From there, you can get as scientific as you want, according to how tight control you have over the different factors. For now, ballpark is just fine and way better than randomly turning the knobs.

Yes, these days we have "HDR monitors" with an absolutely insane contrast range. That's fine in theory and on paper - but absolutely useless for photographic work.

[Imagewerx]

https://forums.adobe.com/people/D+Fosse  wrote

First of all, a TV set is not intended to view or judge photographs. Accuracy is not even on the list of priorities. High quality TV does not mean high quality photo reproduction! Unless you get a decent desktop monitor, or at least a top-level laptop screen, you can't make any judgements. You simply can't. The fact that everything above 244 appears blown-out white to you, is a clear indication that your TV is miles off the map. For one thing, it clearly does not have a standard tone response curve.

But it's all I have!!! I am just a weekend amateur photographer.I go to places,I take photos when I get there to remember how it looked when I'm not there any more.Being able to look at photos I've taken on a screen 55 inches across FOR FREE is a luxury I'd only ever dreamt about until a few years ago.
As I've already said,I have less than perfect eyesight.I've worn glasses for the last 55 years since the ago of four.I have astigmatism.I have photophobia and my eyes are unusually sensitive to red light.
I've said this so many times now,but no one else in the world can see through my eyes,the same way I can't see though anyone else's eyes.

[Todd Shaner]

Imagewerx  wrote

We've had this same conversation on at least one occasion before this.

How does Lightroom build previews when processing RAW images?

Yep, and you're still expecting an uncalibrated TV and cheap Acer monitor to provide accurate color and tone rendering. If that were possible everyone would be using them with no need for expensive displays and monitor calibrators.

Unfortunately it doesn't work as you've stated, "On the white ones I can just about see the checkerboard on 244, anything above that is pure white."  With your current TV (adjusted using your eyes) all highlight tones from 244 through 254 are "clipped" to pure white with no image detail. There's no in-camera setting you can use to prevent these clipped highlights since it occurs after the fact! Inside LR all you can do is lower the exposure to pull all highlight data BELOW the 244 clipping point (see histograms below). This effectively reduces the dynamic range of your raw image data, which is just the opposite of what you're trying to accomplish. The green jag in your last picture edit looks pure black on my calibrated monitor and the histogram shows a heavy-Exposure setting bias.

If you share pictures via email or Web posts they will appear dark with inaccurate tone and color rendering on "normal" desktop, laptop, and mobile device displays. So you and only you can see what appears to be a "normal" image with your LR edits. If that's what you want then we're done here, but I think you can do better!

I totally agree with D Fosse comments, but you mentioned "I don't sell prints or have my photos hung in galleries for others to look at." The best solution is to use a good computer monitor and calibrator OR you can adjust the TV's picture controls by eye using the Lagom or AArdenburg test images files. You should be able to get more accurate rendering with the proper control settings! I also use a Samsung 52" TV for viewing pictures with family and friends, which I've adjusted using the Lagom and ColorChecker test images. The color and tone accuracy isn't as good as my calibrated monitor, but much improved from the factory out-of-box settings.

TV SETTINGS ADJUSTMENT PROCEDURE

Use a "picture" mode with the least dynamic range enhancement such as 'Movie.' Adjust the other settings that I've highlighted in Red below to similar settings on your TV. Most TVs allow making these adjustments separately for each input. Your normal TV viewing settings for broadcast, cable and movie viewing can be different than your computer viewing. I use different settings for my Cable box and BluRay Player HDMI connection, which is also used for JPEG image file slideshows. If the room lighting is bright it's OK to use a higher Backlight setting to make the image brighter. Just make sure you can still see up to about 251 White Saturation level or higher and 3 Black Level or lower. With these settings your LR edits should be quite different!

My Samsung 52" LN-T5265F TV Settings for Picture Viewing

[Imagewerx]

ttps://forums.adobe.com/people/Todd+Shaner  wrote

Imagewerx   wrote

We've had this same conversation on at least one occasion before this.

How does Lightroom build previews when processing RAW images?

Yep, and you're still expecting an uncalibrated TV and cheap Acer monitor to provide accurate color and tone rendering. If that were possible everyone would be using them with no need for expensive displays and monitor calibrators.

TV SETTINGS ADJUSTMENT PROCEDURE

Use a "picture" mode with the least dynamic range enhancement such as 'Movie.' Adjust the other settings that I've highlighted in Red below to similar settings on your TV. Most TVs allow making these adjustments separately for each input. Your normal TV viewing settings for broadcast, cable and movie viewing can be different than your computer viewing. I use different settings for my Cable box and BluRay Player HDMI connection, which is also used for JPEG image file slideshows. If the room lighting is bright it's OK to use a higher Backlight setting to make the image brighter. Just make sure you can still see up to about 251 White Saturation level or higher and 3 Black Level or lower. With these settings your LR edits should be quite different

Not anymore I'm not,I now just use it when I'm typing.My TV set (a 55inch Samsung 55MU6470U) is on more or less those settings anyway.Maybe it's my crappy eyes that are the problem now,or have been all the time?

[Todd Shaner]

Imagewerx  wrote

https://forums.adobe.com/people/Todd+Shaner   wrote

Imagewerx    wrote

We've had this same conversation on at least one occasion before this.

How does Lightroom build previews when processing RAW images? 

Yep, and you're still expecting an uncalibrated TV and cheap Acer monitor to provide accurate color and tone rendering. If that were possible everyone would be using them with no need for expensive displays and monitor calibrators.

Not anymore I'm not,I now just use it when I'm typing.My TV set (a 55inch Samsung 55MU6470U) is on more or less those settings anyway.Maybe it's my crappy eyes that are the problem now,or have been all the time?

The easiest way to determine that is to beg, borrow or purchase a reasonably good monitor and compare it to your TV. You'll need to of course adjust it properly, which you can do using the Lagom or AArdenburg test images files. My vision isn't what it used to be, which is why I invested in a better monitor and a calibrator. Just a suggestion.

[Imagewerx]

https://forums.adobe.com/people/Todd+Shaner  wrote

Imagewerx   wrote

https://forums.adobe.com/people/Todd+Shaner    wrote

Imagewerx     wrote

We've had this same conversation on at least one occasion before this.

How does Lightroom build previews when processing RAW images?  

Yep, and you're still expecting an uncalibrated TV and cheap Acer monitor to provide accurate color and tone rendering. If that were possible everyone would be using them with no need for expensive displays and monitor calibrators.

Not anymore I'm not,I now just use it when I'm typing.My TV set (a 55inch Samsung 55MU6470U) is on more or less those settings anyway.Maybe it's my crappy eyes that are the problem now,or have been all the time?

The easiest way to determine that is to beg, borrow or purchase a reasonably good monitor and compare it to your TV. You'll need to of course adjust it properly, which you can do using the Lagom or AArdenburg test images files. My vision isn't what it used to be, which is why I invested in a better monitor and a calibrator. Just a suggestion.

I really don't know anyone with something like that,so it's unlikely that would ever happen.

I did spend some time calibrating my TV last night using Lagom and got it quite a bit better.The contrast chart now shows me at least 95% of what I should be able to see.The black level only has number 1 invisible to me and white saturation the last 1/2 aren't visible to me.It doesn't look 'right' to me as it's been the other way for so long,so I need to get used to seeing it like this.

And now,in other news.I'd forgotten all about this and what was happening to photos I took last summer when the camera was only a month or so old.I got these wider spaced white spots which I don't get any more,but now seem to have been replaced by the closer spaced green noise spots.
As almost all of my photos are taken outside on sunny days,I didn't see this as a problem and forgot all about it until now.

[Todd Shaner]

Imagewerx  wrote

And now,in other news.I'd forgotten all about this and what was happening to photos I took last summer when the camera was only a month or so old. I got these wider spaced white spots which I don't get any more,but now seem to have been replaced by the closer spaced green noise spots.
As almost all of my photos are taken outside on sunny days,I didn't see this as a problem and forgot all about it until now.

These are "hot pixels," which seems excessive for a 1.0 sec. ISO 800 shot. I noticed this image has a 1920x1080 crop applied, which may indicate you shot this still picture during recording a video. If you were shooting video the sensor gets hot, which can cause hot pixels to appear. It can take five minutes or more for the sensor to cool down after shooting video to prevent this from happening. The green noise is a different issue and I suggest you have the camera checked by Panasonic Tech Support before the warranty runs out. You can refer them to this post for more information.

[Imagewerx]

https://forums.adobe.com/people/Todd+Shaner  wrote

Imagewerx   wrote

And now,in other news.I'd forgotten all about this and what was happening to photos I took last summer when the camera was only a month or so old. I got these wider spaced white spots which I don't get any more,but now seem to have been replaced by the closer spaced green noise spots.
As almost all of my photos are taken outside on sunny days,I didn't see this as a problem and forgot all about it until now.

These are "hot pixels," which is seems excessive for a 1.0 sec. ISO 800 shot. I noticed this image has a 1920x1080 crop applied, which may indicate you shot this still picture during recording a video. If you were shooting video the sensor gets hot, which can cause hot pixels to appear. It can take five minutes or more for the sensor to cool down after shooting video to prevent this from happening. The green noise is a different issue and I suggest you have the camera checked by Panasonic Tech Support before the warranty runs out. You can refer them to this post for more information.

It was taken as as JPEG before I started using just RAW and was cropped quickly to get below the maximum file size for uploads.

[Todd Shaner]

If you were not in video mode prior to this shot then it indicates a problem with the sensor. When shooting JPEG files even with the camera's noise reduction set to OFF you shouldn't see this many hot pixels at 1.0 sec. ISO 800.

[Imagewerx]

https://forums.adobe.com/people/Todd+Shaner  wrote

If you were not in video mode prior to this shot then it indicates a problem with the sensor. When shooting JPEG files even with the camera's noise reduction set to OFF you shouldn't see this many hot pixels at 1.0 sec. ISO 800.

No videos at all that day,just stills.

[Jao vdL]

All TVs come badly calibrated from the factory. Calibration (even just using the charts mentioned above) is essential.

That said, the green noise is not a result of underexposure or anything like that. I have never seen a camera do that even in severely underexposed images. This is a flaw in the sensor or the electronics. Normal behaving sensors will have neutral noise all the way down to zero or only slight tinting. What you are seeing here is either bad internal calibration in the camera with a black level offset in the green that is wrong or a general sensor flaw. I downloaded a bunch of images from https://www.photographyblog.com/reviews/panasonic_lumix_dmc_g80_review/sample_images  and none of th...​in the deepest shadows even when adding +5 exposure and 100 shadows so this is probably your particular copy of the camera unfortunately. It shouldn't be a problem if you don't need to raise shadows too much but as you see here it can be an issue. If you are still under warranty I would return and exchange it. Otherwise you might need to live with this limitation. I would also check if your camera firmware is up to date. This could be a flaw that Panasonic fixed at some stage.

[TheDigitalDog]

https://forums.adobe.com/people/Jao+vdL  wrote

This is a flaw in the sensor or the electronics.

In virtually all the massively underexposed areas of the capture.

[Imagewerx]

https://forums.adobe.com/people/Jao+vdL  wrote

All TVs come badly calibrated from the factory. Calibration (even just using the charts mentioned above) is essential.

That said, the green noise is not a result of underexposure or anything like that. I have never seen a camera do that even in severely underexposed images. This is a flaw in the sensor or the electronics. Normal behaving sensors will have neutral noise all the way down to zero or only slight tinting. What you are seeing here is either bad internal calibration in the camera with a black level offset in the green that is wrong or a general sensor flaw. I downloaded a bunch of images from https://www.photographyblog.com/reviews/panasonic_lumix_dmc_g80_review/sample_images  and none of th...in the deepest shadows even when adding +5 exposure and 100 shadows so this is probably your particular copy of the camera unfortunately. It shouldn't be a problem if you don't need to raise shadows too much but as you see here it can be an issue. If you are still under warranty I would return and exchange it. Otherwise you might need to live with this limitation. I would also check if your camera firmware is up to date. This could be a flaw that Panasonic fixed at some stage.

Firmware is up to date.I didn't ever think this could be a 'fault' so just ignored it.I have a month left on the warranty so will be taking a trip to the camera shop I bought it from later this week.

[Todd Shaner]

https://forums.adobe.com/people/Jao+vdL  wrote

That said, the green noise is not a result of underexposure or anything like that. I have never seen a camera do that even in severely underexposed images. This is a flaw in the sensor or the electronics. Normal behaving sensors will have neutral noise all the way down to zero or only slight tinting. What you are seeing here is either bad internal calibration in the camera with a black level offset in the green that is wrong or a general sensor flaw.

Jao, I opened the heavily underexposed P1080230.RW2 image file in RawDigger with Black Level set to both Auto and Manual 142 142 142 and exported TIFF files. Inside LR the noise looks typical with even distribution of RGB moise pixels and responds well to Color Noise Reduction Amount 25. The original raw file has heavy green bias in the noise and Color Noise Reduction Amount 25 only removes the R & B color noise pixels (see below). This indicates to me the Adobe camera profiles have an issue with the proper black level calculation and not a problem with the camera.

https://forums.adobe.com/people/Jao+vdL  wrote

I downloaded a bunch of images from https://www.photographyblog.com/reviews/panasonic_lumix_dmc_g80_review/sample_images  and none of th...in the deepest shadows even when adding +5 exposure and 100 shadows so this is probably your particular copy of the camera unfortunately.

The green shadows only appear in the OP's raw files when shot with heavy underexposure such as the yellow car P1080230.RW2. None of the RAF files on these review sites are heavily underexposed so no green noise is observed. Does that sound logical?

[D Fosse]

I finally got around to downloading the file. What strikes me is the fact that there is hard clipping in the shadows here. I have never, ever, seen that in any of my own raw files (Nikon D800/D810/Sony a7rII). What should happen is that detail gradually fades away and noise takes over - but you have to dig really deep before you get to pure noise.

But here it's a brick wall. Bam - end of data. That's something you expect in an RGB file, not a raw file.

The green is because blue and red clip before green. Again, that's RGB behavior.

So either the sensor is faulty, or Adobe's profiles are faulty. This shouldn't happen.

[TheDigitalDog]

https://forums.adobe.com/people/D+Fosse  wrote
So either the sensor is faulty, or Adobe's profiles are faulty. This shouldn't happen.

in ACR, Adobe Standard, I see no such green issues with defaults but upping Exposure TOO far does:

But the image is massively under exposed and again, that's the major problem here.

Upping to +1.90, highlights are not clipped, brightness is better, still no noise.

Don't go over board and better, expose better.

[D Fosse]

thedigitaldog  wrote

But the image is massively under exposed and again, that's the major problem here.

It certainly is, I agree with the diagnosis. And you could say that's the answer, and you'd be right.

I'm just puzzled by the hard clipping. Even in a properly exposed shot, there may be parts that received very little exposure and are similarly underexposed as this. That's where I went looking in my own files, and didn't see this hard clipping. What I see is a soft taper towards pure noise, not an absolute clipping point like here.

[Jao vdL]

thedigitaldog   wrote

But the image is massively under exposed and again, that's the major problem here.

It certainly is, I agree with the diagnosis. And you could say that's the answer, and you'd be right.

I disagree. I have never seen massive underexposure do anything like this neon green phenomenon and I have made the mistake of forgetting to reset a -3.0 exposure compensation. That is not how exposure works at all. Underexposure of the entire image has no bearing on what happens in any part of the image except that it simply lets more or less light in. Normally you simply get more noisy shadows if you expose less which is exactly what you expect from the physics of the situation. There are simply fewer photons hitting the pixel and the photon statistics work out to be generally like Poisson noise (i.e. shot noise) which means simply that the noise is proportional to the square root of the light intensity hitting the pixel. You also get some noise from amplifiers there which acts as a noise bottom but recent sensors engineered this almost completely away. When you have areas in an image that have low brightness all that happens is that the shot noise and the amplifier noise start to dominate. There is no preference towards pixels with a green filter in this and all you get is random overall neutral noise. If you get strongly colored noise in deep shadow areas it is any of three things: a defective sensor (mostly the read out electronics have an issue), bad internal software in the camera that tries to do a black correction the wrong way (i.e. subtract a constant value from the image to make the read-out/amplifier noise disappear), or it is bad profiles in the software you are using to interpret the raw files.

https://forums.adobe.com/people/D+Fosse  wrote

I'm just puzzled by the hard clipping. Even in a properly exposed shot, there may be parts that received very little exposure and are similarly underexposed as this. That's where I went looking in my own files, and didn't see this hard clipping. What I see is a soft taper towards pure noise, not an absolute clipping point like here.

Exactly like a camera is supposed to behave. The hard clipping indicates something is done to the raw data either on sensor by a bad offset correction somewhere or in the camera internal software that subtracts a value from the data. It is not supposed to do that. Again it could be a design flaw that affects the whole line of cameras but the absence of the same kind of neon green in deep shadows in other files from the same type of camera. Todd above shows an example of a bit of green noise but this is nowhere near the neon green in the OP's images.

I now remember helping another person on this forum a long time ago that had a Canon 5D that was giving a green haze on every long exposure night image but that behaved normally in day shots in full sun but even there you could find shadows that had green tinting. This indeed turned out to be a defective sensor as that person had it replaced and the problem disappeared. Can't find that thread now unfortunately.

[Imagewerx]

https://forums.adobe.com/people/Jao+vdL  wrote

I now remember helping another person on this forum a long time ago that had a Canon 5D that was giving a green haze on every long exposure night image but that behaved normally in day shots in full sun but even there you could find shadows that had green tinting. This indeed turned out to be a defective sensor as that person had it replaced and the problem disappeared. Can't find that thread now unfortunately.

This reminds me of reciprocity failure with colour film and slides where very long exposures gave an overall green tint to the whole image.

[Bob Somrak]

There are some Green Shadows is one of the RAW photos from https://www.photographyblog.com/reviews/panasonic_lumix_dmc_g80_review/sample_images  and none of th...

so I don't think the OP's camera is any different than other builds of this camera. 

Photo used

Fence in bottom left unprocessed

Fence showing Green Shadows

[Todd Shaner]

https://forums.adobe.com/people/Bob+Somrak  wrote

There are some Green Shadows is one of the RAW photos from https://www.photographyblog.com/reviews/panasonic_lumix_dmc_g80_review/sample_images  and none of th...

so I don't think the OP's camera is any different than other builds of this camera. 

Fence showing Green Shadows

Bob it doesn't look like any LR NR Luminance and Color applied was applied. With it applied there is NO noise. Using the OPs underexposed RAF file example applying LR NR does NOT remove the green noise. See my reply #35 above. I also tested another similar under exposed image at the same site https://img.photographyblog.com/reviews/panasonic_lumix_dmc_g80/sample_images/panasonic_lumix_dmc_g8... and LR NR properly removes ALL color noise. So it does appear there may be something wrong with Imagewerx's Panasonic DMC-G80 camera.

[Bob Somrak]

Thanks Todd Shaner​

OOPS, I forgot that important Noise step