select anchors by dragging while ignoring fills or lines

In the image example there is a cube made with 6 shapes (the background is just a locked image).

Each corner has 3 overlapping anchors from 3 different shapes.

While it is easy to drag-select the 3 outer anchor intersections (marked with green), I am not aware of any way to select the 4th group of overlapping anchors while ignoring any clicks on fills or any line selections. Is there any common practice for this matter?