I'm getting consistent masks with jagged edges with Sky Replacement. See attachments. I've tried various kinds of photos and getting the same results (not just images with angled edges). The photos are approx 8000x5000, RGB files.
Thanks @daleheadrick - Engineering is working on a fix for this for a future update. The workaround, as you noted, is to set the Fade Edge setting to '0'.
Thanks for the reply, Jeffrey. I didn't realize that the extreme stairstepping in a mask's hard edges is actually a result of the Fade Edge setting at its default of 100. Even at 50, there is still quite a bit. So, for any kind of architectual shoot, I will definitely be using a Fade Edge setting of ZERO. 😉
Let's make sure we're in a default state and there are no stale setting somewhere: Restore your preferences using this manual method: https://helpx.adobe.com/photoshop/using/preferences.html#Manually Does it work correctly? If that doesn't solve it, you can quit Photoshop and put the Settings folder back. If it does solve the problem, I would like to get the folder of settings to try and figure out what is bad with them.