Some Supported Hardware Decoded Codecs Not Being Hardware Decoded In Premiere

There are two different forms of compression in H.264.

First, is the 'basic' compression applied to "i-frames" ... a rather common type of data compression technique. This is used on all iframes ... so the All-I versions of H.264 have this applied to them also.

The second type specific to long-GOP, and this is where the H.264 really gets the bulk of the size reduction from. As this isn't "compression", it's ... data mangling, perhaps?

They look at pixels by data ... and if there are several pixels in a small group, that over 2-3-10 frames, stay pretty similar, they simply make them all the same for the entire 'group of pictures' ...

So if you have a four pixel group, 122/106/97, 121/104/99, 123/105/99, and 121/103/100, that stays in similar relationship for several frames, those are simply saved to a matrix table as maybe all 121/105/98.

And you have the iframes, and the matrix tables of either pixels that will change before the next iframe, or pixels that have changed since the last iframe, or ... both.

That is the very different computational task of long-GOP codecs. And requires the CPU to decompress and decode all the frames of a group, plus the next iframe, to play back the second frame of that group.

That's why All-I H.264 is so much easier than 'normal' long-GOP h.264 for the computer.

All-I ... all intraframes ... is similar in CPU load to several other intraframe codecs.

thanks @mayjain7130546 for getting to the bottom of those MP4 files!

I went through and fix both the spreadsheet and the folder of test files and verified that they are what they say they are.

So it seems that for the most maximum compatibility (say for making proxies), premiere is well optomized for H.264/8-bit/4:2:0, H.265/8-bit/4:2:0 and H.265/10-bit/4:2:0. 

Other overvations:

- H.265/10-bit/4:2:2 also is well supported but I'm not sure what "Videotoolbox SW" means in the debug tool for the iMac Pro with the T2 chip. Is this hardware or software decoded?

- H.264/10-bit/4:2:0, H.265/8-bit/4:4:4, H.265/10-bit/4:4:4, H.265/8-bit/4:2:2 are all well supported by the hardware (and also the first three by Nvidia) but chosen not to be implemented in Premiere. Understandable because they're pretty niche combinations.

- Still no idea why my XF-AVC, 10-bit, 4:2:2, Long-GOP footage from my C70 isn't hardware decoded. Not sure what's different about long-gop versus all-intra that would effect this. H.264/10-bit/4:2:2 is normally supported and the XF-AVC version shouldn't be much different. Footage example in the folder!

And here are the files I updated:

libx264_bit10_420.mp4
libx264_bit10_422.mp4
libx264_bit10_444.mp4
libx265_bit10_420.mp4
libx265_bit10_422.mp4
libx265_bit10_444.mp4
libx265_bit12_420.mp4
libx265_bit12_422.mp4
libx265_bit12_444.mp4

Hi @jefftypebeat ,

Thanks for the details. I had a quick glance on the test files you shared and found that 422 10bit mp4 files are actually 422 8-bit files. I believe you will certainly notice HW decode with 422 10bit mp4 files. Please have a look. 
I am also exploring HW usage with other test media and will get back to you. 

Thanks,
Mayjain

thanks for looking into this @mayjain7130546 

1. Here are all my test files: https://shared-assets.adobe.com/link/62dbcbbe-0203-44e8-5e08-e7a62f8c3cb4

These were created using ffmpeg and libx264 and libx265.

2. Link above has every clip in both .mov and .mp4 container.

3. You are totally right. I checked this again on the M1 Max machine and it Hardware Decompressed frames went up under importer.MXF. Correected on the spreadhseet. Photo here: https://share.cleanshot.com/8xwD50Rxc3kGH6gJcLsj

4. I put probably because I wasn't sure how to check for prores hardware decoded. Is there something similar to importor.MPEG? I have no reason to doubt this feature works as advertised. 

5. In the IINA inspector it shows if a video is using the hardware decoder. See here: https://share.cleanshot.com/Bc6tyhFynDqsBDGMYKBJ and here: https://share.cleanshot.com/WpYvwHt2YKmlGR64qrmw

Appreciate you looking into all this!

Hi @jefftypebeat ,

Thanks for your detailed report and highlighting your doubt. Regarding your observations on M1 Max, I have a few questions:-
1. As added in the sheet, 420-10bit/420-12 bit(H264) & 420-12bit(H265) media types are showing HW decode only for mp4 files. Did you try creating the test files in mov ? Can you share these test files ?

2. 422 10 bit files were HW decoded only for mov. We believe it should work with mp4 also.  Can you share the test files where you didn't see HW decoding ?

3. Sony XAVC I is mentioned False. We believe these files are mostly in .mxf container and they should be HW decoded. Please share some media where you didn't see it working. 
4. Are you not seeing ProRes422/4444 files being HW decoded ? The details show 'Probably'.
5. How are you concluding with IINA that files are HW decoded ? Is it by disabling/enabling Hardware Decoding checkbox under video panel and checking CPU/GPU usage from Activity Monitor ?

If possible, please share your test media files https://helpx.adobe.com/creative-cloud/help/share.html so we can investigate the file types quickly and update you accordingly.

Thanks,
Mayjain

Neither Nvidia nor AMD supports 4:2:2 hardware decoding at all no matter what. And Adobe has chosen not to implement 4:4:4 hardware decoding support on Nvidia GPUs (AMD's hardware decoder supports only 4:2:0 no matter what).

That leaves only Intel GPUs (either integrated-on-CPU or discrete) which support 4:2:2 hardware decoding. Please note that only the 11th-Gen or newer Intel CPUs' integrated graphics processors and the discrete Intel Arc graphics cards support 4:2:2 hardware decoding for HEVC.

As for 4:2:2 H.264. you're out of luck. Nobody supports hardware decoding for that format. Only 8-bit 4:2:0 is supported for H.264.

This would take folks like @Fergus H and/or @RjL190365 to sort out.

The table of his link is this: