Pixelated mask while every other layers/masks are the right resolution

I've been working on my mesh for a couple of days and everything worked fine. But now when I add a new layer and add a mask to it, the mask is extremely pixelated. This isn't a UV issue where the UV is too small, because the rest of the layers aren't pixelated. Any guess as to what the problem is? Thanks for the info/help!

I'm using version 8.1.3

On Windows

I don't know the steps to reproduce as it just happened.

Expected result would be for the texture to not be pixelated.