Generative AI over processing the generation

suppose what I need couldnt be uch more simple.

I have a shirt which in this example is green, it has shadows and folds etc as its a real image, I select the shirt using the magentic lasso only and give a command as simple as change the colour to #7182aa, and I end up with images of shirts with all sorts going on, or a denim shirt over the t-shirt, basically everything but just simple changing the colour.

Any advice? My aim is that I want to sell t-shirts, I'm having one t-shirt photographed in a studio, but then I want to change the colour to a specific colour so I have a range.

Check out my result, when all I have typed is - 'change the colour to #7182aa'