I've been using Premiere Pro forever now and probably the last year encoding content into 4k HEVC format in an MP4. One of the things I like about it is how quickly it tracks. Can jump around in pretty much any client app and it immediately starts playing.
So now I have some HDR10 output I'm doing and finding it can sometimes take like in VLC 3 - 5 seconds to start playing again when you jump around. My basic settings:
Preset: 4K UHD (then change...)
Click Match Source
Profile: Main 10
Level: Tried 5.1 and 6.2
Checked Rec 2020 Color Primaries
Checked Include HDR10 Metadata
Set Luminance Min to 0.005
Set Luminance Max to 4,000
Set Content Light Levels Max to 4,000
Set Average to 1161
Did a CBR output to 15Mbps
Quality to Higher
Have also tried with and without checking Use Maximum Render Quality and Render at Maximum Depth.