Rotobrush often gets itself into a state where it is constantly propagating after any alteration

I've been using rotobrush in after effects for a few months now, so I'm still an absolute beginner but I feel I have got to grips with the basics and have been getting some very satisfactory results. 

I am trying to follow suggested workflows as much as possible, so I am trimming my clips to only the parts that need roto-ing and keeping them short (only a few seconds) and I have the clip being roto'd alone in its own precomp.

A lot of the time it works fine, but sometimes, for reasons I can't put my finger on, the rotobrush tool gets itself into a state where ANYTHING I do with the rotobrush... foreground paint, background paint or change the frame I'm on, causes rotobrush to re-propogate the whole clip from the beginning. It becomes impossible to work with.

I don't *think* I'm doing anything wrong, and I can find examples of other people with the same problem in this forum, but it never seems to be acknowledged as a problem and always implied that it's user error. 

I am 100% open to it being my user error, so my questions are: when this constant propagation happens, what could I try to fix it? Does anyone know for certain something you can do to trigger this behaviour so I can avoid it?

Many thanks.