How to track mask tracking the edge, not the enclosed are?

Let’s say I have a clip of a building and want to mask the windows. Both the building’s wall and the reflection from the window are moving (not a static image). If I track the mask for a window, the mask follows the movement of the reflections from the window, not the window frame.

So how could I make the tracker follow the window frame(s)?

Thanks in advance!