Why does Roto Brush keep reselecting unwanted areas? How can I improve the accuracy?

Question about the Roto Brush in After Effects

I'm having some questions about how the Roto Brush in After Effects determines the selection area.

What criteria does it use to decide what gets selected? I assume it considers factors like color, edges, and object recognition, but even when I explicitly exclude certain areas, it sometimes still re-selects them after a few frames. In some cases, the same area keeps getting selected no matter how many times I try to correct it.

Even when working on relatively simple areas like a face, I often find that if I render and walk away, the brush has ended up selecting completely unrelated parts of the frame.

Would adjusting the contrast, color, or sharpening the edges of the subject help improve accuracy?
If there are any effective ways to improve Roto Brush results, I’d really love to hear them.

I’m also using third-party AI-powered effects that allow prompt-based masking. These apply selections across the entire clip instantly, which is very fast compared to the Roto Brush — which can take 100 times longer. Why is there such a drastic difference in speed and performance between these two methods?

If anyone with knowledge or experience can shed some light on this, I’d truly appreciate it.