Zoom Tool Shortcut Switches Back to Previous Tool

Hi - When I'm in one tool (any tool), then switch to zoom by pressing z on my Mac, it switches to zoom, then quckly switches back to the tool I was in. This happens randomly, so not every time, but sometimes. It results in me clicking on a spot of the image with what I think is the zoom tool, but an erasor or healing brush or something else I didn't expect. Then I have to undo and come bsck with the zoom. It's a sucky time waster - very frustrating.