I've run into this before and it's really annoying. Changing the version of PS doesn't matter - they all do this but FWIW I'm using the desktop CC versions 2020/2021 -
I do a lot of pixel level editing with high reolution A7RIV files - sometimes we need to do a head replacement (for instance) - but when transform+fractional rotation is applied the jaggies kick in really badly I get this doing composite work as well. There usually isn't a lot fo resizing going on - the attached samples were relatively same size materials.
So far the solution is to go over the jaggies with a clone brush from each side, and smooth them out. Guess what I'm wondering is why can't the image turn smoothly? It's not the borders of the copied sections either - it's any clean smooth lines of contrast in the center seem to get crunched. Yes, I'm zoomed way in - but the jaggies weren't there on the original copied section after it was pasted - only upon being rotated.
Please post more meaningful screenshots – so please don’t crop them and include all the pertinent Panels.
Is there a change after committing the transformation?
What is the Image Interposation setting?
OK - the first one is a copy paste of a head - and the settings.
Has not been rotated - looks fine.
The second one is the zoomed in view to confirm.
The third one is with settings - after initiating a transform rotate - but not applied yet (preview)
The fourth is after applying it.
Interpolation is Nearest
Interpolation is Nearest - but forgot to mention changing this to any of the other values as an experiment didn't make any difference.
»Interpolation is Nearest - but forgot to mention changing this to any of the other values as an experiment didn't make any difference.«
Did you restart Photoshop?
Why did you choose Nearest Neighbor to begin with?
Yes of course PS has been restarted mant times, upgraded - different versions tried (as originally mmentioned) - all the same. I just tried it again on the very latest version downloaded yesterday. Same consistent result.
Nearest happens to be the default - but like I said no difference regardless. For giggle I just also tried rectangualr marquee select/copy/paste/rotate vs lasso - same issue.
Attached - latest version of PS bicubic smoother applied instead - not quite the same but still a rubbish result.
»Nearest happens to be the default«
No, it’s not.
Your screenshots are not useful – can you please stick with one magnification (ideally 100%) and always include all the pertinent Panels and Bars?
Or can you simply provide the file with the rotated and unrotated Layer or better Smart Object?`
Re. resizing (and maybe also the algorithms you are using), Nearest Neighbor AFAIK does NOT add anti-aliasing — which may be the reason you chose it at one point. However, this can make straight lines look jagged (aliasing), so it's more for images of nature and such.
However, if you do 50% resize on an anti-aliased image with Nearest Neighbor, the result should be a good compromise (if you prefer NN) (acc. to what I've heard; haven't tried myself).
If you work at this pixel peeping level, IMO you should know the characteritics of these modes very well.
From the nerds I've also heard PS is often not using the very best algorithms for this kind of stuff, so in the end you may still be right to a degree 😉