Weird Crop tool behaviour with 16-bit files

[erickellerman]

Shadow areas of my 16-bit files show strange patterns if I use the Crop tool to rotate. I can also replicate the problem by doing the following:

Make a new empty 16-bit file in PS6 in RGB mode (File>New). Fill it with 100% black. Use the Crop tool and rotate to taste. Press Enter to accept the crop and zoom in to enjoy the black and grey noise patterning reminiscent of a man's suit. The Straighten tool will achieve the same result. Your suit pattern will vary according to the degree of rotation.

The problem goes away if you subsequently convert the file to 8-bit, or convert to any other mode (Greyscale, Lab, CMYK).

This patterning is easily visible at 100% view or less, and appears whether the View>Show>Pixel Grid option is on or off. It is decidedly NOT the pixel grid I am seeing.

This effect can be created with files from both my Canon and Hasselblad cameras.

I'm using a Mac Pro with OS10.6.8.

[Noel Carboni]

Paulo or anyone experiencing the visibility of the transparency glitch...  Does transparency otherwise seem to work normally?

It's almost hard to picture how it could, but I'm guessing it must or more major problems than this would be reported.  It's almost as though the LS byte inaccuracy is getting combined into the MS byte.

-Noel

[conroy]

Noel Carboni wrote:

Paulo or anyone experiencing the visibility of the transparency glitch...  Does transparency otherwise seem to work normally?

It's almost hard to picture how it could, but I'm guessing it must or more major problems than this would be reported.  It's almost as though the LS byte inaccuracy is getting combined into the MS byte.

-Noel

Yes, transparency is normally OK. The glitch seems only to be a display artefact involving blending of the Transparency Grid with partially transparent image pixels.

[Paulo Skylar]

Noel Carboni wrote:

...  Does transparency otherwise seem to work normally?

Noel,

A qualified yes! That is, nothing has presented itself that suggests any other problem with transparency. On the other hand, I have not run through a series of tests looking specifically for transparency problems.  Also, I have seen no transparency issues, including the one that is the subject of this thread, in CS4 on the same hardware with the same drivers.

I may take a closer look if time permits.

Paulo

[conroy]

Also, I have seen no transparency issues, including the one that is the subject of this thread, in CS4 on the same hardware with the same drivers.

I get the glitch in both CS6 and CS5.1 (Ps 12.1).

[conroy]

Chris Cox wrote:

You're showing low bit errors -- that is a minor problem, made visible only by extreme adjustments.

We'll get that fixed when we can.

This topic is talking about a major problem that makes low bit errors visible on some GPUs when they should not be visible.

We need to get a reproducable case of this so we can figure out where the bug is (in Photoshop, in the driver, etc.) and get it fixed.

OK, I didn't realise that you'd already determined that some resampling is creating errors in low bits of transparency values in 16-bit mode. That's why I presented an extreme adjustment of the transparency values extracted to an image.

Yes, here the transparency errors are visible without adjustment when affected layer dark pixels are viewed over a bright Transparency Grid. I'll get more info together.

[Chris Cox]

Try changing the GPU level in preferences, I'll bet the artifacts disappear in basic but show up in normal or advanced (posts 51 and 53).

Then we need to know your GPU and make sure you're using the latest driver.

Then from that we can try to reproduce the error.

[conroy]

Chris Cox wrote:

Try changing the GPU level in preferences, I'll bet the artifacts disappear in basic but show up in normal or advanced (posts 51 and 53).

Then we need to know your GPU and make sure you're using the latest driver.

Then from that we can try to reproduce the error.

That's right, the tiny transparency errors do not result in visible artefacts when the GPU level is "basic".

GPU is NVIDIA GeForce 320M in a mid-2010 Mac Mini running OS X 10.6.8 Snow Leopard with whatever driver is the most recently provided by Apple for that system.

[Chris Cox]

It's weird that the error shows up in multiple GPU brands, but not with all driver versions.

It still seems to be in the shaders that convert color for the display, but I'll have to check with the GPU guys to see why it is so hit and miss.

[Noel Carboni]

FYI, I've just updated to ATI Catalyst 12.6 released drivers, and now I see the slight transparency glitch expressed without extreme enhancement.

Summarizing:  Catalyst 12.2 did NOT do this, 12.6 DOES do this.

-Noel

[Chris Cox]

>> Catalyst 12.2 did NOT do this, 12.6 DOES do this.

Two steps forward, one step back. Sigh.

[Paulo Skylar]

There seems to be yet another problem, or a related one, with the 16-bit mode in CS6. Rather than put this observation into

a new thread it will remain in this one as I happened across it while dealing with the transparency issue discussed above.

I have been visualizing the problem in the transparency channel using layer blending modes rather than by using extreme adjustment layers - not for any good reason other than it seemed a bit easier for me to do.  My recent favorite procedure to create and visualize the problems of16-bit interpolation artifacts and those artifacts spilling into the transparency channel  has been to create a 100x100 pixel document and fill it with 50% gray. Using the image size  command the document is upsampled to 550x550 pixels and that larger image looks normal to the unaided eye - uniform medium gray.. That layer is then duplicated (Ctrl+J) and its blending mode is changed to hard mix. Voila, you have a dramatic rendering of the 16-bit mode problems. as shown here

The new problem is that I do not think that the layer math should be revealing the problem.  If I execute the same process in 8-bit mode I get an all-white result, which is what I expect it to be. To eliminate the interpolation and transparency issues I made a new document 550x550 pixels, filled it with 50% gray, did NO scaling, duplicated it and changed the duped layer's blend mode to hard mix. The result is an all black image - again, not what one would expect.  If I look at each of three component channels they are each all white, but the composite channel view is black. What's that all about ?

So the 16-bit mode in cs6 has interpolation problems, transparency channel problems and blend mode problems.  I am not going to search for any other issues nor will I just happen upon any other malfunctions as I am now finished using 16-bit mode in CS6 until these issues are resolved, hopefully in the first update.

Paulo

[erickellerman]

I can confirm your finding, Paulo.

[Paulo Skylar]

Erickellerman,

Good, it's always good to know one's results are not unique .

Paulo

[Noel Carboni]

I think that's because the new Photoshop default for resampling is Bicubic Automatic.  Worst idea Adobe ever came up with.

Try changing your default upsampling scheme to just plain Bicubic, Paulo.  The problem goes away entirely.

As to why the channels show white instead of black, it's pretty clear they're using a cache level because they're so small, which causes the math to switch to 8 bit for layer compositing.  You want to eliminate THAT disparity, change the cache levels to 1 and GPU mode to Basic, which causes all layer compositing for displays to be done in the full 16 bits.

-Noel

[Hudechrome]

Since the problem resolves by going to 8 bit, and since I deliver my work to the client as 8 bits, it doesn't make a whit of difference. Also, printing is still 8 bit so I don't see any problems there either.

I did generate the pattern as outlined by the OP. I am running an Sapphire 7750, driver vers 12.6. I can see the problem clearly just eyeballing it at 100% The quality of the artifacts appears to be related to the Straighten angle; at 16.8°, it appears as an interferometer pattern. Changing to 8 bits clears it up, and turning off the GPU in Prefs also eliminates the problem.

I know that there are still driver problems or at least the driver seems to be able to be corrupted if I use Sleep Mode. That has been fixed by reinstalling the driver and turning off Sleep. (It is a desktop, not a laptop).

There seems to be no difference between 12.2 and any other driver for this card, afaik, at this time.

I am eyeballing the black image only at this time. No tweaks to exaggerate the patterns.

Edit: FYI, I'm running a PC, Win7 64 Ultimate.

The PC consists of a Asus M4A77TD mobo, Athlon IIx4 630, cpu stock speed,12G Ram

Video Driver version atiumdag 8.98.0.0

Message was edited by: Hudechrome

[Hudechrome]

Looks like I am the contrarian here.

Paolo, I cannot reproduce your effects. The image here is the 550px upsample with the settings shown superimposed over the black supposedly crosshatched set to fill screen.

I've run the test over and over with no difference in outcome. Use Graphics Processor is checked, Advanced Mode . Cache set at 4.

[Noel Carboni]

8 or 16 bits, Lawrence?

Was the fill color 128, 128, 128?

-Noel

[Hudechrome]

Interesting.

1) 16 bit

2 ) Fill color set to 50 using LAB, corresponding to 100, 100,100.

However RGB in Color Settings set to ProPhoto

If I set it to sRGB and use RGB to set gray, at 128 128 128, now I get pure white.

In fact, I get pure white with a variety of RGB settings except for 100 100 100, where it is pure black, ProPhoto or sRGB. The correspondence to L in Lab changes, depending on the Color Settings.

You can check this easily by selecting each fill layer  Thumbnail on the layers palette and  proceed to change both layers to the same value. It's interesting to set each layer differently as well, but the only outcome to be pure black has 100 100 100 in either layer. IOW, if 100,100,100 is set on only one layer, the other can be anything and I get black.

The bottom line is I get wither pure white or pure black. No crosshatch.

???

[Noel Carboni]

Are you using Bicubic or Bicubic Automatic for the resampling?

If Bicubic Automatic, I guess I need to get one o them new fangled 7000 series video cards then. 

-Noel

[Hudechrome]

It would be interesting to see if you validate my results after doing so.

Tom's HW gives it a best buy and it doesn't require aux pwr connection. Runs at about 55W full throttle.

[Noel Carboni]

I've actually got my eye on the 7850, Lawrence. 

-Noel

[Hudechrome]

I did too, even the 7950, but then I saw the price....

I paid $109 for the 7750.

[Paulo Skylar]

I finally got around to dealing with your posting Noel, sorry for the delay.

For the example I presented you have found a good workaround in choosing bicubic for resampling. My goal, of course, is not to find work arounds but to generate more information to help Adobe fix any related issues. Also, if I changed the example process from scaling up the mid-gray square to let's say transforming the square to a 45 degree rotation of itself (everything else being the same) then the problem is present even with bicubic resampling for the transformation.

Changing the cache levels to 1 is not  that good a work around for me as it completely turns off caching and thus affects the performance of other operations. It's a good piece of diagnostic information that you have uncovered and I hope it quickens an adobe repair. In the meantime I remain wary of using 16-bit mode in CS6.

Paulo

[Noel Carboni]

Actually I don't consider the choice of Bicubic as a workaround, since I consider it a better method for resampling in general than either of the two "newer" methods (ESPECIALLY Bicubic Sharper, which adds horrendous sharpening artifacts).

I do hope Adobe gets to the bottom of this display inaccuracy issue though (which is apparently display driver-specific).  It's not nice to be deceived by the document display.

Given that the AMD folks are not releasing new drivers very often any more, it'll probably be a while before we see a newer driver for ATI cards, and I want to believe that Adobe will be able to find a different way to use the GPU facilities so as to avoid these visible inaccuracies.

My intuition tells me that the document quality itself is not adversely affected by the use of 16 bits/channel mode - that this is just a display issue.  But I respect your wanting to be wary of it.

-Noel

[MikeKPhoto]

Chris, just as an FYI A see the same issue with the Intel 4000 driver latest but not the Beta Version. I do not see the "suit" but the same artifacts in the second image as shown in Post 38.

Hope this helps you track the issue

Cheers

MK