Draggers problem/recording

Hi, I encountered a problem while recording a dragger move. I have a puppet that have multiple views (frontal, right, left and back profile). When I drag and record move of lets say right wrist dragger in frontal view it also picks and  records movements of dragger from other profile views although I record in only one view. Then in timeline I see four handles recorded instead of only one. When I pick and drag a handle it obviously picks also the draggers that are in approximate postition, but in other profile views although not shown. I have profile views  activation via the swap set so its only one view possible at the time. Any solution to that problem?