P: Feathered layer masks don't render correctly in Classic or macOS Preview

[#1]

I have reproduced a bug forwarded to me by a student, using a second machine.  Some problem occurs when PSD files are saved from Photoshop including non-trivial layer masks with the feather layer property set.  it is difficult for me to tell whether the PSD file is corrupted or whether the PSD file is misread by multiple other applications.

Steps to reproduce:
1.  Load an image in PS.
2.  Add an adjustment layer (I used brightness/contrast, and set brightness down to make the effect observable.)
3.  Set the layer feathering property on the adjustment layer to a non-trivial value, say, 50%.
4.  Paint on the layer in such a way that the mask itself is non-trivial.
5.  Save the result as PSD.
6.  Load the image into any of a variety of applications (including LR Classic and OSX Preview), note the image looks as if the layer mask was improperly computed.  

Notes:
A.  I have reproduced this bug with different images.
B.  Both I am my student have reproduce this bug--each on different MacBook Pros running Catalina.
C.  The student has reproduced this bug both with graphics acceleration enabled and disabled.
D.  Images open/display correctly in Photoshop when loaded directly
E.  Images open/display correctly in Photoshop when loaded via LR's "Edit Original", even when the image does not display correctly in LR.
F.  I have not looked at the issue via any other operating system.

Catalina 10.15.1
PS 21.0.1 Release
LR Classic 9.0 Release


(edited to add omitted step for setting layer feathering property.)

[J453]

In Photoshop, go to Preferences > Performance... and check "Legacy Compositing" - resave the file. Does it render correctly in the other apps?

[FeedbackCommunityMember]

Based on your latest release of Adobe Photoshop Desktop 21.0.1, the "Feather" function of the "Layer masks" is not honored once you save the file and it's rendered in Adobe Bridge or Capture One. 

Any changes you made to that layer, with the Layer Mask "Feathered" will render as if you never applied a Layer Mask at all. In essence, using the "Feather" feature will render your Layer Mask useless (once you save the file and view it outside of Photoshop).

[J453]

I can't reproduce this off-hand (only have access to 10.14.6 at the moment). It might be Catalina only so will try when I'm back in the office. If you can post an affected file, that might also help. (CC-files or Dropbox)

[#5]

More details and tests later, but here is a subtly affected file.  You should see 11/20 written in the sky, when there is a bug, the 0 and just a tiny bit of the 2 are not rendered.

Let me know if you have trouble with the Dropbox link

https://www.dropbox.com/s/7nkynjk25v1zuc1/1120.psd?dl=0

[johnrellis]

I was just troubleshooting a PSD compatibility-layer issue with Catalina's Quick Look, so I looked into this. 

The new compositing engine in PS 21 incorrectly omits an adjustment layer with a feathered mask from the compatibility layer (which other apps use to preview the PSD). The legacy compositing engine in PS 21 correctly includes the feathered adjustment layer in the compatibility layer.  This occurs with the graphics processor enabled or not, though the image using the graphics processor is a little garbled.

PS 20 always includes the adjustment layer with a feathered mask, regardless of compositing engine and graphics processor.

So the workaround is to enable Preferences > Performance > Legacy Compositing, as Jeffrey suggested.

Here's a folder with all combinations of feathered/unfeathered, legacy engine / non-legacy engine, graphics processor / no graphics processor in PS 21, with two combinations from PS 20 as well:

https://www.dropbox.com/sh/7azync5qqykskfg/AADYT9mc_xiGPYkO7CvDcmMpa?dl=0



Note that on Mac OS 10.15 (Catalina), if the PSD wasn't saved with a compatibility layer (Maximize Compatibility), then Finder / Quick Look will pick one of the non-adjustment layers to display -- it won't do a faithful rendering of all the layers.  (Prior versions of Mac OS would display the file as all white.) So make sure you always have Maximize Compatibility set.

[J453]

Thanks!

[J453]

Thanks, John.

[#9]

Can confirm that switching on legacy composting resolves the issue.  Thank you!

[#10]

That did resolve the problem, thank you!

[FeedbackCommunityMember]

I've experienced the same issue.  Any photo where I use feathering on a mask for an adjustment layer has odd issues when opened in LR.  It's not simply that the feathered layer is ignored, I seem to get some kind of rectangular area where the layer adjustment is applied, and another area where it is not.  This shape doesn't seem related to the actual content of the mask pre-feathering.  Changing to legacy compositing fixes the issue.

[Sam Breach]

I am experiencing this issue too. I am still on High Sierra (old machine). Thanks for the tip - switching to legacy compositing fixed it for me too. 

[J453]

Hi Sam, have you updated to 21.0.2? That should fix the issue without having to turn on Legacy.

[johnrellis]

I just retested the bug recipe in PS 21.1 / Mac OS 10.14.6 and it failed.  It doesn't fail after turning on  Preferences > Performance > Legacy Compositing.


I've changed the status back to No Status.

[risottto]

Ok, I posted this once in the Main Adobe Photoshop forum and it was instantly flagged as spam, which is ridiculous, it's a major BUG!

I have a TIFF I worked on in Photoshop, mostly cleaning up some edges and at the very end I applied a Hue/Saturation layer to the top, clearly masked to only the wall, which was blue because of tape on the side of the work.

The image looks totally fine opened up in Photoshop, (Right image below), but when viewed in any other program (Quick Preview here on the left), or in Lightroom it is the same, and exports with the glitch in the image!

The TIFF from outside of Photoshop has a clear arbitrary rectangle where the Hue/Saturation Adjustment Layer is not applied properly. A random rectangle is drawn over the image, and the adjustment is applied everywhere except for that rectangle, completely unrelated to the actual layer mask!

I opened up the file again, changed the name of the top layer, re-saved, and it seemed to fix it, but that is definitely not normal behavior.

Has anyone else had this problem? It seems like a major program error, a file appearing one way inside of PS, then completely butchered when viewed or exported from anywhere outside of PS!

[johnrellis]

As a workaround, enable the option Preferences > Performance > Legacy Compositing and resave the file.

[risottto]

Yes!!!! Looks like Legacy Compositing does make a difference, replicated the OP's scenario by feathering and painting into an effect layer mask, (a daily task for me), and it properly displays in LR 9.2 (Using PS 21.1.0 on Catalina 10.15.2).

With Legacy Compositing off, the Layer mask does not get applied properly in LR or Apple Preview, but not rectangularly glitched as in my file.

Have not been able to replicate the rectangle glitch with Legacy on, and sticking with PSD with compatibility mode instead of multi layer TIFF.

I would think using the very latest versions of LR and PS subscription they should be compatible with each other without reverting to legacy mode!

[johnrellis]

"I would think using the very latest versions of LR and PS subscription they should be compatible with each other without reverting to legacy mode!"

It's purely a bug in how PS computes the hidden compatibility layer that LR and other apps like Preview use to view PSDs. It has nothing to do with LR per se (thankfully).

[FeedbackCommunityMember]

I've an image that I did some work on in LR, and then moved to PS. When done, the saved image appears with a dark rectangular banding in LR but not in PS. I was led here by this answer.

Please fix this bug!



See the area marked by a red rectangle below.

[J453]

This issue was fixed in 21.0.2 and later. John Doe, what version of Photoshop are you using? Can you post a copy of the PSD you're having trouble with?

[johnrellis]

I tested the bug recipe a few weeks ago in 21.1.0 and it failed:
https://feedback.photoshop.com/photoshop_family/topics/reproducible-bug-in-feathered-layer-masks-pho...

I retested it just now in 21.1.1 and it still fails. 

[FeedbackCommunityMember]

21.1.1 solved my problem.

[johnrellis]

Here's a screen recording showing 21.1.1 failing on the original bug recipe:
https://www.dropbox.com/s/aziud3ko5nax1vs/photoshop-feather-mask-bug.2020.03.20.mov?dl=0

[J453]

Thanks John, sorry for the late reply. I'll get this taken care of.

[J453]

Fixed 21.1.2 or later.