Select Subject creates unwantedy thin masks on edges of other subjects

[scotwllm]

Maybe my English isn't very good, but when I load a selection and I tell Photoshop to stroke it, I expect the stroke to replace the marching ants. I certainly don't expect Photoshop to stroke other selections as well.

It's not even consistent with which extra stuff it outlines. In the attached, it drew in the line my leg and shoes, head and shirt, and arm crossing my torso. It leaves out where my legs and shorts meet as well as shirt and shorts.

Why does Photoshop do this? How do I make Photoshop stop doing this?

Scott

(thread converted to bug report and title edited by moderator)PEC

[Bojan Živković]

Test like this: use lasso to create a selection similar to what you have. Create a freeform selection using any tool but do not use a selection method converted from a continuous tone image.
Apply Stroke, if extra lines appears then try with preference reset https://helpx.adobe.com/photoshop/using/preferences.html

If no extra lines appears then everything is fine, when you are creating a selection from an image where some pixels are not fully selected thus no marching ants appears. When you use Stroke it fills "non visible" selection which appears as not fully opaque lines.

[Mylenium]

Nothing wrong. You are combining individual selections based on antialiased masks. In the computer world, 0.5 + 0.5 isn't always 1 in terms of transparency and that's why you get the artifacts with the partially selected pixels as already explained.

Mylenium

[Trevor.Dennis]

Try hitting the Q key to toggle Quick Select on.  That will show areas with less than 50% selection which would not be enclosed with marching ants.  In fact Quick Select is the ideal tool for inspecting selections.

[PECourtejoie]

Hello, thanks for the provided steps, but a crucial one is missing: how did you create the selection you are pasting?

It seems that you had multiple levels of selection, and that the stroke was applied on those lines.

[scotwllm]

I started by selecting the subject. Then I selected the subject's components (head, shirt, arm, shorts, legs, shoes). I understand about neighboring selections and why you end up with a transparent(ish) line between them. I expand whichever selection goes on top by 1 pixel. So anyway, I select the shoes and intersect the selection with subject and save the selection as shoes. Then I quick select the legs, arms and head, intersect it with subject, subtract the shoes, and call it skin. Then I select the shirt, intersect it with subject, subtract skin, etc., continually refining the selections and edges until I have the various components. To check my work, I add the components one by one, and it should equal the subject. What I wanted to do was to stroke the outline of each selection with 1 pixel of black on a white background to look for gaps or overlaps. In the screen shots I provided, I selected the subject, which should have just been the outline of all the parts. You should not have been able to see the internal parts because I did not select them.

[scotwllm]

It did it again. I filled the "arms" selection with a skin tone, and it outlined the shirt (which was not selected).l

[PECourtejoie]

Pinging @CoryShubert about thin lines in selection issues. 

Also adding this thread: https://community.adobe.com/t5/photoshop-ecosystem-discussions/more-schmutz/m-p/13550134#M704384
Do you have any info on the camera that created those files? anything done, like sharpening prior to the selection that would make select subject fail?

[Jqqerry]

@PECourtejoie  When you watched the video you left the link to, I saw that sometimes happens when I checked the layer mask using the selection tool in Photoshop.
For example, that's going to choose something called A, and there will be times when other parts will be included. And if you exclude it from selection, it will definitely have to be excluded. But it leaves some solid lines.
This is the From Select Subject, Object Selection Tool, and Quick Selection Tool all of these tools.
(In particular, there is no problem in adding a selection area with the Quick Selection Tool, but such solid lines appear very often when subtracted.
To prevent that, you should brush it off directly in Lasso or quick mask mode.
It's a pretty old phenomenon. I don't know if it's a bug, but it leaves a very thin solid line, so I don't really see the selection. That means it appears to be neatly excluded from the layer mask. However, people who paint like this go through it more clearly because they can see that there is one only when they check it themselves.
Among those retouching images, you can see a lot of such solid lines if you change the color of your clothes rapidly (white clothes to black clothes).
So the best and clean way was to check and erase the pen tool or layer mask every time.)

In short, I don't know if this is a bug.
But it's old.
No matter what tool you use, you don't have to "add" the selection.
But you already have a selection, and you don't like it, so you have that solid line problem when you use the tool to "subtract the selection."
There is no other way at the moment than to erase that solid line directly with Lasso or with a brush directly from the Layer Mask as I wrote above.
Photoshop's Selection Tool created a small solid line, but it doesn't recognize it and we can barely see it. But the solid line is left.
That phenomenon doesn't show an error when you create a selection and do a very small adjustment (using level or curve for Brighten 2-3%. I mean, there's no problem.
But if you're strongly calibrating or overwriting a certain color like painting, you'll experience that error yourself.

And at least I remember experiencing that phenomenon during Photoshop 2015. It's not just the latest version.
In particular, the Quick Selection Tool leaves a lot of solid lines like that, so there is also an opportunity to change my workflow by using the pen tool more.

[PECourtejoie]

There are several issues at hand, and I think that the first one is the object /subject selection tool selecting parts of the other person.

another is selecting feathered areas, but it seems that most of the issues would be alleviated by using a large rectangle and filling the entire area, instead of trying to fill a feathered a selection.

unless you were using this case as an example of other struggles.

[PECourtejoie]

When you invert a feathered selection, parts that are 70% grey will be selected at 30%, so it is almost impossible to just revert and erase, unless you crunch the selection using levels. A lot could still be done on selections, but IMHO, the selections should be better to start with. I am sure the Ps team would love seeing your images and the improper selections/masks created, if they could be shared.

[D Fosse]

If you ask me, all of this goes to show that you can never trust automatic procedures and expect a perfect result. I'm not saying this to be smug, only to be realistic. This is all just pixels. Photoshop doesn't work with "objects", it doesn't "know" what the picture shows, beyond what AI-algorithms have been trained to. But the algorithms have never seen this image before!

Making good selections is difficult. You need to put some work into it.

[PECourtejoie]

Dag, the OP complained initially of other issues, like stroking producing unwanted appearances, but I decided to convert this to a bug, as another of their threads had the same root cause: if there was not extraneous selection, they might not have struggled to edit them.

[D Fosse]

OK, got that.

But I noticed the OP said: "I expect the stroke to replace the marching ants". As we both know, that's not how it works, and that is a faulty expectation. An area 49% selected will not show marching ants, but still be 49% selected.

Trevor has the best approach to this: hit Q to check the selection as Quick Mask. I routinely do that. If there are stray areas, just paint them out.

Here's an (extreme) example just to make the point: This is a 1 pixel inside red stroke on a luminosity selection. The image itself is black-and-white image. This is not a red mask! This is only the 1 pixel stroke, nothing else:

[PECourtejoie]

Rest assured that we are on the same sheet of music, @D Fosse , indeed, the default marching ants selection preview can be deceiving. IMHO, all the preview modes available in Select and Mask should be available for all selections.

(I wish there was a shortcut when in Quickmask mode to hide the composite, and view only the quickmask channel in B&W)

[scotwllm]

The selections started as quick selects, but I used the pen tool around the edges, adding or subtracting from the selections to make them as precise as possible.