PS renders weird strokes

Right, that solves the problem. Could convert to pixels later and then not touch it anymore.

I apologize if some terms are wrong, but I use the german version and "Kontur füllen" is best translated by me as "Fill stroke".

I didn't speak of layer styles, but when I draw a circular selection, apply 1 px of stroke, then add a layer style that adds another 49 px of stroke, the result is the same. I think this  is because the 1 px fill already created a non-circular ring. Different method: draw a shape and set fill to transparent. You get the same 1 px ring. Question: how to create a perfect ring in PS?

I have a windows system with Photoshop 6 running on it. The code for layer style strokes hasn't changed since that feature was introduced.

No. I didn't save the yellow on white example from above.

But I remember to have never seen this before with thin strokes like 5 px or 10 px. I work with Photoshop since version 7 (good ole times, it would start within a second on today's machines running a VM). I'm not sure if I have ever used a 50 px stroke before, but what I definitely noticed over the past years is that rendering those selection masks has deteriorated gradually. I also posted another topic a while ago when I wanted to create round edges on a rectangular selection and used the "Smooth edges" tool and it was very much not as expected. There I also put a comparison. I still have a purchased CS6 on disc, but cannot install it on Windows 10. I really would like to compare it with an older version of Photoshop. Maybe I download the CS6 and install it as demo in a VM. If there really is a difference, I'll post here.

I believe the Stroke layer effect has always behaved this way. Do you have a PSD file you can share that exhibits the behavior? (Use CC files or dropbox to share a link to download the file)