Poor Quality Transparency When Pasting From Clipboard

[ThioJoe]

In recent versions you are able to copy images from the programs like the browser and paste it into Photoshop, maintaining transparency. Which is greatly appreciated but...

Problem: Semi-transparent pixels in images pasted this way do not look correct. I know why it's doing this I think, more on that later.

Example: This emoji: https://raw.githubusercontent.com/googlefonts/noto-emoji/main/png/128/emoji_u1f642.png 

How the edges should look. This is how it looks if pasted into MS Paint for example, or if the file is saved and imported.

How it looks when pasting into Photoshop - notice the black background from the BMP leaking through.

Cause TL;DR: I'm fairly certain this is because Photoshop is using the BMP (bitmap) format of the image in the clipboard, despite the original PNG format being available also in the clipboard.

Context: Looking with InsideClipboard after copying an image, you can see when copying a PNG image from Chrome, it puts both the BMP version and PNG version into the clipboard.

(Note: CF_BITMAP, CF_DIB, and CF_DIBV5 are all the bitmap image in slightly different formats, Windows basically automatically creates the other two if any of them are put into the clipboard. Not sure which photoshop is pulling from.)

 

 

Cause Details:

BMP doesn't natively support transparency, so Windows allows a pseudo-transparency in the clipboard for BMPs where the BMP clipboard items are basically are the image and mask data concatonated together. But presumably the transparency is flattened to black, so even though there's the mask, semi-transparent pixels aren't going to look right.

Solution: 

Have it check for other image formats in the clipboard before pasting, and only use the BMP version as a fallback.

[Lumigraphics]

Does Windows allow dragging images directly to the Desktop? On the Mac, you'll get a saved PNG file doing that.

[ThioJoe]

Yea I you could always do that but it's just the extra bit of hassle.

I think the important thing is the issue isn't immediately obvious if you don't zoom in, so someone might not notice it isn't the original quality until down the line, or assume that's just how the image was.

[Stephen_A_Marsh]

I tested on a Mac and this appears to be an issue only on Windows.

What you could also do is: Layer > Matting > Remove Black Matte or Defringe

Yes, like the suggestion from @Lumigraphics these are workarounds, as a user that is all that I can suggest outside of the Photoshop development team changing how the clipboard content is accessed on Windows.

[Sameer K]

Hey, @ThioJoe. Thanks for sharing the details. I've requested our team to review and comment on this. Here is a simplified version of what they shared. 

Clipboard transparency involves maintaining alpha channel information, which requires advanced data handling. On macOS, the clipboard natively supports formats that include transparency, such as PDF and certain image types. While Windows clipboard data often defaults to formats like BMP, which do not support transparency, achieving clipboard transparency on Windows is possible through third-party tools or custom solutions. Recent Windows updates have improved clipboard features, including cloud sync and history, though full transparency support is still developing.

I hope this clarifies. 

Sameer K

(Use '@mention' to tag me when you reply)

[TomPinkerton]

Recently (a few months back), Photoshop began recognizing the CF_DIBV5 clipboard data format when present. This format supports 8 bit alpha. However, I suspect there may be confusion as to whether the image colors should be interpreted as premultiplied with this alpha.

Photoshop currently assumes the colors are not premultiplied, but after a little digging it appears some applications (like Chrome) may place premultiplied image data on the clipboard for this format. The darker semi-transparent fringe colors in the posted example image could be explained by this interpretation of premultiplied colors as non-premultiplied (the colors would appear darker/closer to black than they should). Note that this does not appear to be an issue of transparency being flattened to a 1-bit mask, as the final pasted image still contains 8-bit semi-transparent pixels.

If this is true, Photoshop could change its decoding of CF_DIBV5 clipboard data to assume premultiplied colors instead. But if this is not a convention honored by all applications we may not be able to rely on this as a finished solution. In this case we might need to do as you suggest and recognize PNG data as well and make it a priority over the CF_DIBV5 clipboard format.

We still need to investigate this further to confirm things, but I'll file a ticket to resolve the problem. Thank you for the detailed report.

[ThioJoe]

I think you're right about the issue being premultiplication. I found a couple discussions like this with people talking about it with chrome specifically, but it doesn't seem consistent app-to-app: https://stackoverflow.com/questions/15689541/win32-clipboard-and-alpha-channel-images

I tried digging around in the chromium source code, and even though it is enormous I was able to find this line part of the "PNGCodec::Encode" function that gets used to convert the image data to the PNG format for the clipboard (which is down stream from the code that seems to start the clipboard copying, specifically following the "EncodeBitmapToPngAcceptJank" function):

format == FORMAT_SkBitmap ? kPremul_SkAlphaType : kUnpremul_SkAlphaType;

 
The "sk" seems to be related to the "skia" library for graphics. From what I've gathered, even if the user right clicks and copies a png file, it doesn't actually copy the original original PNG file to the clipboard, but rather creates some other image object - even if it's a jpg for example. And that line above is part of the function that prepares the PNG data that will end up on the clipboard. 

So basically what I assume is going on is the transparency is premultiplied when placed in the skia object, and judging by that code it checks whether it needs to unpremultiply based on some stored property in the skia object. But it seems that the DIB_V5 version that gets added to the clipboard does not get this treatment (it seems to create the DIB_V5 image data in this function).

Though maybe it stores an original unmultiplied version instead of doing an operation to do so, since from what I was reading, premultiplication is lossy, yet the pixel colors from the PNG data from the chrome clipboard seems exactly the same as the original image when saved (despite the compression and stuff being different). 

Edit: Actually I just found this other section in the code comments where it specifically talks about "FORMAT_SkBitmap" preferring premultiplied alpha. I think that format is the object type from which other formats are converted.

Anyway regardless it seems chrome probably is doing premultiplication which is affecting the clipboard bitmap version. Apparently the PNG specification requires data not be premultiplied so it should be most reliable to use as the source if available on the clipboard I would think.

[Lumigraphics]

Does the Windows clipboard even support PNG? I'm not a Windows developer or even a Windows user by choice. They could support PDF which has transparency...

[ThioJoe]

Does the Windows clipboard even support PNG? I'm not a Windows developer or even a Windows user by choice. They could support PDF which has transparency...

By @Lumigraphics

Windows clipboard supports any arbitrary formats. You can add multiple "things" / formats to the clipboard at once, each identified with a number and a string name. There are a handful standard clipboard (like bitmap and text), but an application can also "register" their own format which basically just means add one with a custom name.

The PNG format would be a registered format. Technically you can add whatever data you want to a registered format, so you could even fill a clipboard format called "PNG" with random data if you wanted, there isn't any kind of check. You could even add a format called "blahblahblah" and fill it with hundreds of hundred megabytes of zeros.

So anyway it's not that the windows clipboard wouldn't support a format, but instead it depends on each application supporting it by looking at the available formats currently on the clipboard, and then hoosing to parse ones called "PNG" for example, because it's pretty universal that any format with that name will have the same contents as a proper PNG file.

The only reason I know all this is because I've been working on a hobby side project for messing around with the clipboard: https://github.com/ThioJoe/EditClipboardContents

[Lumigraphics]

Yes it supports arbitrary formats but only with bespoke tags. I'm talking generic built-in support. Because if applications don't put PNG on the Clipbhoard, Photoshop can't paste it.

[TomPinkerton]

Excellent analysis, and thank you for the deeper investigation. This is very useful information.

The right strategy for Photoshop will likely be to first look for PNG and use that if it's present. But if PNG is not available we should fall back to CF_DIBV5 next if it's available.

For CF_DIBV5 we should probably interpret the colors as pre-multiplied since that appears to be the more common convention for that format (though some applications may still export it as non-multiplied).

As you point out, pre-multiplying the colors is lossy, so it would be better to use the non-multiplied colors found in PNG data if it's available (assuming those colors weren't themselves reconstructed from a pre-multiplied source - but of course that's out of our control).

[Lumigraphics]

https://www.codeproject.com/Reference/1091137/Windows-Clipboard-Formats

https://developer.apple.com/documentation/appkit/nspasteboard/pasteboardtype

[FranciscoNelson]

Hello Team,

I work heavily in Photoshop and PowerPoint. For the past decade, I have been able to drag items from Photoshop to PowerPoint and it has always preserved the transparency mode. I was also able to simply COPY and PASTE from Photoshop to PowerPoint with no issues. I never had to export individual items as PNG. 

This allowed me to move extreamly quickly and get presentations done extreamly fast. Fasforward to the past few months, I am no longer able to do that. Everytime I try, I there now appears just a blank square, or a whitebackground around the item i copied.

I find myself exporting so many layers and saving so many items that i will never use again. This has slowed me down by hours. Because I now need to export so many layer and then upload them to PowerPoint. In my position, we are presentation driven. I NEED THIS BACK. 

However, whenever I work from Illustrator, the feature still works perfectly fine. With no issue, therefore, this is an error or something thats not working within the Photoshop platform. CAN someone guide me to a better solution.