Whenever I go to move something in XD the wiggling of the mouse caused by accidental scrolls makes it almost impossible to place something where I intended. For example, if I try and move a box to align it with my guides it will start to jitter because my finger is moving slightly on the magic mouse causing small scrolling in every direction, often leaving my object a few pixels off of where it should be. Can scroll be disabled when you are moving something, I dont rememeber this being an issue in photoshop.
Sorry to hear you are having trouble. We are unable to reproduce the issue because we donot have the full set up to test. We would request you to please connect with us directly using this link: https://helpx.adobe.com/contact.html so that we can help you on 1:1 basis.