If you are trying to use the Erase functionality, please note that the default Brush is "Quick Select" which automatically tries to select related segments of the image. Sounds like it is being too aggressive in its selection.
For a more precise (but manual) selection, you can switch to the Circle brush as shown below