option+command+clicking the layer used to subtract those pixels from your selection, however, it appears this shortcut has been updated with "zoom to layer". It still subtracts those pixels, but zooms out every time, which really interrupts my workflow.