Transparency "edge case" (pun intended)

[OsakaWebbie]

For a book cover, I have created the images in Photoshop but assembled the final thing in InDesign to work with the text more easily. A small part of a scifi character overlaps the book title to add depth, so I need to keep the background graphics and foreground graphics separate so that the title layer can be between them. I didn't think that would be a problem. But the semi-transparent fur of the character, which has highlights to indicate rim light from glowing objects in the background, ends up with dark edges in InDesign (and the resulting PDF), and I can't figure out why or what to do about it.

Originally I was saving a TIFF file from Photoshop, but after I noticed this problem, I placed the Photoshop file (with only the needed layers for this character) directly in InDesign. I even tried merging all the layers first, in case there was something about the darker base layer that was coming through the rim light layer due to the reduced transparency in that area of the image. But even after doing that, it still looks fine in Photoshop (and in TIFF, if I use that method) but not in InDesign or any exports from there. At one point I thought the cause might be that the Photoshop image is 600dpi but the exported PDF reduces the images to 300dpi, but even if I don't downscale in the PDF export, the dark edges are still there. Thoughts? If I should be posting in the Photoshop community instead, for tricks at that stage to prevent this from happening, let me know, but I thought I'd try here, since it's in InDesign where the problem appears.

Here are screenshot snippets of the same zoomed-in area (the back of the alien's head, as he stands in a dark room but looks out into space with bright stars and other stuff that cast light on him) in three apps. Look at the left and right sides of the head.

First, in Photoshop:

Next, viewed in InDesign:

And finally, in the resulting PDF viewed by Acrobat Pro:

[Brad @ Roaring Mouse]

My first thought was also that i was an affectation of the downsampling, as that WOULD potentially allow allow a sliver of the background black show through at the edges, but since you also tried no downsampling, then I'm stumped. Let me think on this a bit. I can see that viewing it in InDesign would do it, as your screen preview is downsampled and anti-aliased, and depending on your Display settings in Acrobat, the same might be happening.

Have you tried opening the PDF in Photoshop to render at say 1200? Is the "edge" still there?

Can you share a chunk as a sample image?

[OsakaWebbie]

Opening the PDF in Photoshop, yes, the "edge" is still there. I don't know what you mean by rendering at 1200 - I can't imagine upscaling being of any use.

As for a sample image, the Photoshop file with layers is too big, but attached is a crop of the TIFF. Try putting it in InDesign with a medium-lightness background, and I assume you'll see what I'm seeing.

If I can't solve this in InDesign or with something simple in Photoshop, I guess I'll have to try to trim back the edges of all the underlying layers (a base color layer that is mostly a reddish brown, a shadow layer, and a couple others with details), even though that's a lot of work and risks creating other oddities. Even though I can't see those layers' color coming through the rim light layer, perhaps something is going on under the hood.

[rob day]

HI @OsakaWebbie , Can you share the InDesign file and the placed images?

[OsakaWebbie]

A couple of the files are too big for this forum's attachment limits, and I doubt it would help anyway.

[Brad @ Roaring Mouse]

The test at 1200 was meant to replicate how the file might be handled handled at an actual RIP on an output device, which would be anywhere from 1200 to 3600 dpi... Just to eliminate the possibility the anomaly was just affecting viewing the file in InDesign or Acrobat.

[BobLevine]

I would do the entire thing in Photoshop, including the text and then save it as a Photoshop PDF. Placing that in InDesign will retain the vector properties of the text. TIFF and PSD will rasterize it.

[rob day]

Sorry if I’m misunderstanding the problem, but if I add a solid gray behind the figure layer I get this:

[OsakaWebbie]

@rob day I hadn't tested bringing the TIFF back into Photoshop, but I guess it does the same thing as InDesign in that case. While still in Photoshop with the original image layers, that dark edge isn't there. (I was mistaken when I said I can merge layers and be okay - apparently that's not quite the case.) After creating a base layer with what Aaron Blaise calls "local color" (including the fur at the edges), I press Ctrl and click on that layer thumbnail to select the not-transparent area of that layer before painting other layers (shadow, highlight, rim light, etc.) Perhaps that doesn't quite work right with semi-transparent pixels. This project has taken me way deeper in Photoshop than I've ever been before (I even bought a pen tablet just for this purpose, even though I'm not really an artist).

@BobLevine I'm much more comfortable in InDesign than Photoshop - I wouldn't know how to do the text treatments I'm doing if I had to use Photoshop for everything. (I'm sure it's all possible, but I struggle.) And it's convenient to have multiple pages in one document, because there are actually three book covers, not just one, with some common elements I can put on the parent page.

Only a small part of the image overlaps with the book title (the ends of the alien's long ears), so perhaps I should just export a complete background image including the aliens (which would look clean like my first screenshot above) and then put an image of only the ear tips on top of the text, which would be far less noticeable. I lack confidence to try to reduce the edges of the darker layers in Photoshop, partly because of my lack of skills (I worked really hard to get this far) and also because I would not get feedback while I'm working on it (since the problem doesn't appear while still working on layers in Photoshop). Thanks for trying to help.

[rob day]

Can you Package and share the InDesign file and assets?

[BobLevine]

Okay then, here's another idea. Create the layered graphic and save it with layer comps, place it twice, one on top of the other using the layers options and put your text between the two images.

[OsakaWebbie]

I don't know what you mean by layer comps, but I tried this: First I tried saving a TIF with layers (insanely large at over 600MB), but I couldn't select layers in InDesign - it seemed to just be a bloated way of having the same thing as a flat TIFF of everything. So I went back to Photoshop, removed everything related only to the other two books (to reduce the file from almost 2GB to 450MB), and placed that in InDesign, selecting only the layers related to the aliens. But the result was the same. See the screenshot here - the head on the right (and the background) is the layered TIFF - nice and smooth because the background and foreground are together. The head on the left is the Photoshop file with only the alien layers selected during the place operation.

 

I gave up doing it the "proper" way. Today I exported one TIFF of the whole scene and another TIFF of the aliens with transparency, and in InDesign only use the tiny portion of the transparent image that overlaps the book title.

[rennie_0973]

I've run into this kind of edge artifact before when using layered PSDs with soft glows or rim lighting over dark bases. Even if nothing looks off in Photoshop, InDesign and exported PDFs can reveal unexpected blending behavior, especially around semi-transparent pixels. One thing that helped me was flattening only the rim-lit character onto a solid neutral-gray background, then setting that gray to be knocked out or keyed in InDesign. Essentially you're faking transparency without relying on feathered alpha edges. It’s a bit of a hack, but it avoids those dark halos while still preserving detail.

[Peter Spier]

For a book cover, I have created the images in Photoshop but assembled the final thing in InDesign to work with the text more easily. A small part of a scifi character overlaps the book title to add depth, so I need to keep the background graphics and foreground graphics separate so that the title layer can be between them. I didn't think that would be a problem. But the semi-transparent fur of the character, which has highlights to indicate rim light from glowing objects in the background, ends up with dark edges in InDesign (and the resulting PDF), and I can't figure out why or what to do about it.

By @OsakaWebbie

Do you see this if the text layer is hidden? Transpaency above text can be problematic for the text for sure, and perhaps the problem is extencing into the background.

I tend to agree with @BobLevine  here that it might be better to do the entire composite in Photoshop (at least any part that has image overlapping text -- text on a layer above the image should be OK to add in InDesign). With waht I presume will be fairly large type for the title I can't see that it would be much of an issue.

[OsakaWebbie]

@Peter Spier For people good at Photoshop, doing it all in PS makes sense. I originally started out planning to do it all in PS. But I didn't know how to get text the way I wanted it (gradients, effects, etc.) and was getting tired of googling every little thing and watching tons of tutorial videos. So I fired up my beloved InDesign - I can save named swatches of colors and gradients (and if I modify a swatch, the content follows), and I know where to find all the controls and effects. Plus, the Photoshop document was getting unwieldy. Even without the text, it has a ton of layers that are challenging to organize (I have all three books in one document because of common elements), and even though I've changed all the large embedded images to linked, the file size is barely below the 2GB limit for PSD (I'd never even heard of PSB and didn't want to try it).

In answer to your first question, yes, the problem exists even without the text. I ended up using a flat TIFF of all the image content plus another TIFF just for the bits of alien that overlap the text - it looks good enough.

[rob day]

One thing about Photoshop transparency—if you have a single layer with transparency the preview is as if the background below is clear—an analogy would be you are printing on acetate. If you add a layer filled with white below the semi-transparent edges change. I’ve attached you PSD with a solid white layer below:

[rob day]

This shows it more clearly

[rob day · Accepted solution]

The Photoshop transparent to opaque gradient placed in InDesign over a white filled rectangle with Overprint/Separation Preview turned off:

InDesign with Overprint/Separation Preview turned on:

Default PDF/X-4 Export:

[OsakaWebbie · Accepted solution]

@rob day That's very illuminating - I suspect that is why I looked at the TIFF file with transparency and thought it was okay. I decided to do the hack workaround - I exported one TIFF of the whole scene and another TIFF of the aliens with transparency, and in InDesign only use the tiny portion of the transparent image that overlaps the book title.

[rob day]

Photoshop has the Minimum and Maximum filters which let you expand or contract a layer mask. 

[rob day]

Attached is an example of Minimum set to 3.0 applied to the layer’s transparency mask, which clips the dark halo