The problem is a buggy video driver.
Select and Mask runs in OpenCL, a component in the video driver. This is one of those functions that use the GPU to do data processing, and the result returned to Photoshop for further processing.
Update or roll back your driver.