Export creates silence at beginning; audio shifts

Apparently h.264 doesn't support uncompressed audio, so when exporting, silence is added to the beginning of the audio track and from what I can tell, the whole track is shifting that amount and cutting off the end audio to fit within the video frames.

I've tried other suggested codecs that support uncompressed audio such as ProRes, but the file is five times bigger, the video quality isn't noticeably better and playing the videos in IINA (like VLC, but better) still leaves a jarring glitch when going from clip to clip.

Specifically what I'm trying to do is gapless playback for videos. I've created videos for audio tracks that in iTunes play back to back with no audible silence (aka gapless playback). It's like one continuous medley, but each song is it's own track so that they can be rearranged on the fly. Before iTunes implemented gapless playback I had to prerecord our 3 hour long sets (in real-time!) before every show (2-3 times a week!). iTunes made it possible to make the sets on the way to the gig and even rearrange them WHILE we were playing (while saving my sanity). Now we want to add videos to the pre-recorded tracks and have that same flexibility.

I tried cutting out the silence in Audition, but after exporting, the silence is back, albeit half as long. I also tried adding a frame and 576 samples (also tried 1024) to the Pro Tools bounced WAV (then I cut off the silence in Premiere) but it was still shifting and/or squashing the audio when exporting, so it didn't make a difference.

I'm not even sure what I'm asking for...maybe a codec that doesn't squash or shift the audio? Maybe someone knows of a video player that allows you to make playlists AND play continuously without a gap (I'm looking at  you, VLC). My understanding is that  iTunes gapless works by removing the silence at the beginning of mp3 and AAC tracks. Seems like there would be a video player that does the same by now. It's 2020.

Bronze lining: at least if I have to pre-render the whole set, it only takes a few minutes (this takes me right back to the days of moving tracks around Pro Tools and then waiting an hour for it to bounce to disk). Conceivably I could re-render the second set on our set break if I wanted to make changes. But come on technology! We're supposed to have flying cars by now!