How to use sticking tool without affecting other layer?

I had put the sticking tool & draggable on the arm of character. At the same position I also create a swap trigger to show different gesture. When I drag or move the default arm it will affect the trigger arm & it become distortion. Is it possible to sticking without affecting hidden layer or trigger layer? I tried grouping & ungrouping the layer. The result is just same.