how to remove source scrubbing with wireless mouse

Is there a way to remove scrubbing in the source monitor while using a wireless mouse?  On my wireless Mac magic mouse, when my finger moves/swipes left and right, the playhead scrubs left and right. BUT, it is very sensitive, so while I am playing a clip, my finger will move just a bit on the mouse, and the playhead will start scrubbing, so I either have to move the pointer off of the source monitor completely, or try as best I can to not move my finger while the source is playing. 

It's a minor problem, but annoying when it happens all day.  I might have to switch back to my wired mighty mouse if I can't figure this out.

Thanks.