Straighten and Content Aware Fill ?

I have an image which had alot of perspective; but Sampler did an alright Job along with crop and the perspective tools to get the image straightened. 

Although because the image had alot of perspective, parts of the color map; along with the other maps have still that perspective. How can I remove it, is there a way to which I can straightend out those areas by masking them out use a perspective tool then sorta like a content aware fill or clone I can fill in the other areas so that it doesn't look as if it adjusted in that area but yet those areas straighted out ?

Otherwise if this is not possible; I suppose hopefully Photoshop. I just thought of maybe doing most of the clean up in sampler.