Seems like seemingly simple things like changing the color of house isn't possible or obvious

I uploaded a picture of my house into the generative fill tool.  I selected the house and in the prompt tried a number of variations of paint my house red and it couldn't figure it out.  Seems like it just can't register what the house is.  How do I create a prompt that just changes the one or two attributes highlighted in the prompt?  Or is something just broken?

I have a screen capture of my experience...but too not sure how to share a .mov here.