All the XAVC codecs use long-GOP compression, so they're hard as Hades on an NLE to begin with.
And it's what, a 4k 10 bit file at what's the framerate ... 100fps?
Yea, that's a load right there. Proxy the stuff. Note, your proxies need to be in an intraframe codec for best playback. Like ProRes, Cineform, or a DNx variant. Which means they'll be much larger on disc than your original clips, but vastly easier on your system for playback.