Blocky artifacts when using a position map - Help!


I'm having a strange issue with my position map, when applying a texture to my model that uses the map it creates these blocky artifacts all over the texture.
I'm baking my maps in Marmoset, and I've tried messing with the settings and nothing has worked. This is testing with only the position map applied, it makes no difference if I add all the other maps.
I've linked what the artifacts look like in the paint window and the position map that I am using from Marmoset.
Any help would be appreciated!
