Anchor points in PS: Deleting via context menu vs. deleting via 'delete' key

This question has been bugging me for years, it just took me this long to ask it (since I almost never deal with vector).

When using the pen tool to create paths that then become fill layers... and then making changes to that new shape layer... what's the difference between right-clicking an anchor point and selecting Delete Anchor Point versus selecting that same anchor point and hitting the Delete key outright? Because the two methods seem to lead to very different behaviors.

For one thing, if I select a bunch of anchors and try to delete them via the (right-click) context menu, it only deletes the one I'm hovering over. The Delete key, however, allows me to delete them all at once. But this method seems to lead to what can best be described as "wonky stuff" (my best guess is that it leaves the overall shape unclosed, with shapes extruding where they shouldn't).

I tried holding down the usual shift/ctrl/alt keys (in various combinations) while hitting that Delete, hoping by some miracle this would mimic the right-click Delete's behavior, but no such luck : every attempt broke the shape.

How do I remove a bunch of anchor points quickly without touching the Delete key that seems to break the shape?