Inconsistent results when using keyboard shortcuts

I'm using photoshop with wacom display and am getting some very odd behaviors with the keyboard shortcuts.

I'm toggling between drawing and erasing quite often and am using the B and E keyboard shortcuts to do so.

Usually B and E switch the tool mode from Brush to Eraser. I often need to maintain the brush size between the tools and I've read the the ~ keyboard shortcut does this. It seems though when I use the ~ the B and E keys now only temporarity enter the tool mode for as long as I hold the key.

Is there a way to have the B and E keys only switch the mode and the ~ maintain the brush size without totally changing the behavior of the brush and the eraser tools?

Utterly confused as usual.

Noah