Having trouble with a RGB File: error message and unable to change in to a greysdale image

An error comes up that the tool I want to use will not work because the quick mask is not turned on. I dont think I made the layer into a quick mask. How do I undo this? Also the image mode is greyed out and not functioning. Thank you in advance.