H.264 (AVC) mp4 Hardware vs Software Encoding

When I use Adobe Media Encoder to export h264 mp4 files using the Hardware Encoder, I noticed a gamma shift on playback that was really driving me insane. After a little sleuthing, I determined the cause was due to the resulting file not including the following information that had been present in the source mov and mp4 files:
- Color range
- Color primaries
- Transfer characteristics
- Matrix coefficients

The difference was easily visible when I used After Effects Render Queue to export ProRes 422/422HQ/4444 mov files...these all render and play back as expected, so ProRes had been my workaround for uploading to various websites (including YouTube, Facebook, SmugMug).

As a quick test, I switched to the H.264 Software Encoder and, sure enough, the expected data gets properly encoded in the mp4 files:
- Color range : Limited
- Color primaries : BT.709
- Transfer characteristics : BT.709
- Matrix coefficients : BT.709

These mp4 files now play correctly in every player I have tried them in (VLC, Windows Media Player, Media Player Classic, Chrome) and on every website I have uploaded them to except SmugMug (I reached out to their support...will continue to upload ProRes clips to their website until there is a resolution). In other words, I have a valid workaround...just slightly slower since I can't take advantage of the hardware encoder's speed. I have a nVidia RTX2070 currently using Studio driver 472.84.  I don't know if this issue is due to how the hardware encoder is implemented on Adobe's end or if it is an issue with the hardware vendor's provided driver...but it IS an issue that has caused much grief over the past several months!

The only difference between the Software Encoder color values and the source files (from my Canon EOS R5) is that the camera encodes clips with "Full" color range. It would be nice if there was an option for the h264 encoder to use "Full" vs "Limited" color range, but it's not a big deal as long as the players handle playback correctly.  YouTube converts full color range clips to limited anyway.

Based on Adobe's documention, I get the sense there's a bit of confusion regarding how their own products handle color management. This passage in particular is a red flag: "Enabling color management is useful when you want to display the color appearance of a timeline on a reference monitor. Disabling color management is useful when your screen matches the media on the timeline. It works well for Rec. 709, sRGB, and social media delivery." As mentioned, my source clips are in Rec 709 (BT.709) Gamma 2.4. I'm on a PC, so I expect the "working space" to display the clips as sRGB IEC61966-2.1. Adobe color mangement appears to have this part configured and working correctly. As long as the resulting clips are output as Rec 709 Gamma 2.4, they SHOULD display correctly on YouTube because their player EXPECTS Standard Dynamic Range clips to be Rec 709 Gamma 2.4. This may not have always been the case, but it definitely is now.

That said, if the color range, color primaries, transfer characteristics, matrix coefficients info isn't present, the clips will display incorrectly even using PC players (I suspect they just pass through the data straight...hence why they appear brighter and washed out). So, I could use the hardware encoder with clips that have already been converted to sRGB, but that opens up a can of worms. What if the player defaults to Rec 709 and you load a clip that has been rendered in sRGB? It will get darker and more saturated!