Audio drift in 3 hour export only with h265?

I've got a 3 hour long film that I need to send to some clients in a compressed form so I want to use h265 instead of my usual h264 export. But when it's completed the audio tracks both shift over the 3 hours? This doesn't happen when I export as H264.

If it helps it also has 2 audio tracks that both shift away from each other as well as the footage so none of it is remotely in sync by the end?

Both audio tracks are from identical recorders with identical settings (both 4800hz) and the video is all shot at a set 25fps on a Lumix S5ii.

Is there something obvious I'm missing here?

Currently attempting to export it all as a massive pro-res file to see if converting that then to h265 makes a difference.