Enable Additional Hardware Accelerated Decoding Support for H.264 and HEVC Footage

Report · Dec 01, 2023

I did some research recently due to a massive playback issue not too long ago with some HEVC footage I recorded with OBS Studio (for some context, the OBS footage was constant frame rate, not VFR).

Here is the link to that post for reference:

https://community.adobe.com/t5/premiere-pro-bugs/hevc-8-bit-4-4-4-footage-is-very-hard-to-playback-i...

To sum it up, I realized that one of the main causes for my playback issues was with Adobe's lack of support for hardware accelerated decoding for a majority of the flavors of H.264 and HEVC codecs.

Here are a couple screenshots as well as links to the articles from Puget Systems back in December, 2022:

Premiere Pro: https://www.pugetsystems.com/labs/articles/what-h-264-and-h-265-hardware-decoding-is-supported-in-pr...

DaVinci Resolve: https://www.pugetsystems.com/labs/articles/what-h-264-and-h-265-hardware-decoding-is-supported-in-da...

For some context, here are a few specs from my PC setup:

CPU: Ryzen 9 7950x 16-Core, 32-Thread

GPU: NVIDIA RTX 4090

RAM: 128GB of DDR5

My GPU is an RTX 4090 and even though it can decode video with NVDEC in all flavors of H.264 and HEVC codecs (including AV1, which Adobe currently doesn't support as of this post), it doesn't get used due to Adobe's lack of support for the footage I'm editing in (HEVC 8-bit 4:4:4 in MOV and MP4). And so that task is given to my CPU.

However I have an AMD CPU, a Ryzen 9 7950x, and unfortunately AMD doesn't put their Video Core Next hardware core in their CPUs; unlike Intel, which puts their Quick Sync Video hardware core in their CPUs. So really, the video decoding on my system becomes software only instead of hardware accelerated.

This makes the editing experience in Premiere Pro extremely difficult to manage with poor video playback in my timelines to the point where it's sometimes impossible to even work without transcoding to another codec.

For now, I've been transcoding to ProRes proxies to get around my issue.

Suggestion:

I highly recommend Adobe enable more hardware accelerated decoding support for the different flavors of H.264 and HEVC codecs.

While I understand that a lot of people are often editing 8-bit 4:2:0 footage with these codecs, having support for only this heavily overshadows editors and filmmakers that are recording in higher quality flavors of these codecs either because they want to or they simply can't afford to use ProRes due to high file sizes.

Other editing platforms, such as DaVinci Resolve, currently have more hardware accelerated decoding support than Premiere Pro (as seen from the chart above) and so I would really appreciate it for Adobe to really up their game on this.

I love Premiere and so I'd love to see smooth playback in my editing timelines for more types of footage!

Report · Dec 01, 2023

A 2022 list is a bit old. One or two completely new versions may have shipped that chart was new. And Adobe is at times not so quick to update data.

So @Fergus H ... would have the most recent information.

Report · Dec 02, 2023

Some of what you requested is completely impossible – and it's dictated by the hardware manufacturers themselves. Absolutely nobody on the Windows side of things supports, nor will they ever support, hardware H.264 decoding outside of 8-bit 4:2:0. What's more, neither AMD nor Nvidia will ever support hardware acceleration of 4:2:2 of any kind. Thus, only Intel will support 4:2:2 (at least for the foreseeable future).

Report · Dec 02, 2023

When it comes to what software (ie. Premiere Pro) receives hardware decoding support for certain kinds of codecs, is it really determined by the hardware manufacturers?

Their GPUs are proven to handle these different flavors of H.264 and HEVC codecs. Does Adobe have to contact these hardware manufacturers in order to unlock their GPUs' ability to decode those types of footage? Or is it based on the software application instead and that it's really Premiere Pro that needs to be able to unlock more flavors of these codecs on the GPU?

In that case, I'm a little curious as to how DaVinci Resolve was able to gain more hardware decoding support than Premiere Pro.

Forgive me if I'm clueless. I'm just trying to better understand the big picture here.

To me, it seems a little backwards to limit hardware decoding to only 8-bit 4:2:0 mainly because most modern cameras nowadays (both mirrorless and cinema cameras) are able to record much higher quality than just 8-bit 4:2:0 (and creative professionals are demanding for higher bit depths and chroma subsampling). I feel like Adobe should be making room for more support as more people are using these higher quality flavors of H.264 and HEVC for their projects.

I understand adoption can be very slow due to other hardware manufacturers and software developers needing to update their tech in order to make room for those new codecs (not mention some needing licensing like HEVC). However, some of these codecs have been around for many years. Adobe being one of the companies heavily versed with different video codecs because of Premiere Pro, I feel they should have included more support than just the bare minumum of 8 to 10-bit 4:2:0 by this point in time....especially when their competitors, like Resolve, now have more hardware decoding support than them.

AMD and NVIDIA I believe should be aware that a good number of their customer user base are creative professionals that rely on their GPUs to complete their projects. For them to not support hardware acceleration of 4:2:2 or higher I feel is very discouraging if that really is true. Video creatives are investing in their products because they expect them to handle various kinds of footage at least somewhat seamlessly.

I get it if the current demand for higher quality flavors of these H.264 and HEVC codecs isn't quite high as mainstream audiences are used to viewing content in 8-bit 4:2:0 in SDR. I'm not asking to change that since I understand that's most likely not going to change anytime soon.

I'm mainly asking for more support on behalf of video editors, VFX artists, filmmakers, and content creators that are working with these codecs constantly. Not everyone is filming in ProRes and not everyone has the storage for it either. I think it's great that we have the option to use more efficent codecs in 10 to 12-bit color and 4:4:4 chroma subsampling, but they need to be easier to work with and playback in editing timelines so that the editing experience is much better and creatives can complete their projects faster.

It does sound like a very tall ask (one might call it a pipe dream even).

However, after my tedious playback issues I feel the lack of hardware decoding support in Premiere is getting pretty out of hand and it's slowing down my (and possibly others') editing process.

Hoping there is at least some form of a solution soon.

Report · Dec 03, 2023

Like I stated in several discussions, Nvidia does support 4:4:4 HEVC hardware decoding (at least on paper). However, Adobe currently locks out 4:4:4 hardware decoding as its implementation of that format still has bugs to iron out.

AMD does not support anything but 4:2:0 no matter what.

Report · Dec 03, 2023

Just wondering why you need to record as HEVC 8-bit 4:4:4 ? This is an extremely compressed format and wondering if you'd be able to see the difference in your output? Not challenging you, just trying to help... (sorry if I didn't need to say this, but some people get very defensive when no offense is intended)... Always useful to test your workflow from beginning to end to see what's necessary to maintain quality... And just to say it outloud, I have very limited experience with OBS...

I just recently did a 2 camera shoot with iphone 14s... Transcoded to prores (while converting to constant frame rate) and then generated prores proxies... Yes they take up alot more drive space but drives are so cheap compared to what they used to be... And easy enough to leave the computer working overnight to generate the files..

Report · Dec 03, 2023

I'm with Michael on this. Very puzzled by choosing to work in 444, but only 8 bit. As someone who works for/with/teaches colorists, I hearcacton about quality of differing media options. And I've never heard of anyone working 444 with only an 8 bit clip.

So like Michael, just ... puzzled. Would love an explanation.

Report · Dec 03, 2023

Thanks for the vote of support Neil. Not always easy to remember to ask why
the OP is asking the question, not just answer it.

Report · Dec 03, 2023

No worries! Yeah, I understand. (Also sorry, I have a habit of writing a lot.)

The reason I chose to go with HEVC 8-bit 444 I'll admit is a pretty niche reason. But it ultimately comes down to storage capacity and the medium I'm capturing footage in.

I'm not recording live-action (otherwise, I would definitely be filming in ProRes). It's computer generated content rendered in real-time in the Unity game engine. Virtual production, essentially, but not high budget.

Think of it like the animated movie, Surf's Up, where they filmed their scenes using physical rigs to create an authentic documentary look to their camera movements in the 3D space. That is basically exactly what I'm doing as well as a small community of other people doing this.

To capture this footage, I use OBS Studio.

The majority of the content rendered onscreen is in the sRGB/Rec.709 color space, which is basically 8-bit color. It's user generated content so there isn't much creative control over the color space coverage (and my monitor is sRGB only as well).

OBS Studio can record all flavors of ProRes using the FFmpeg encoder. However, recording in 10-bit color would mostly be diminishing returns due to the nature of the stuff we're filming. And also, OBS didn't have the best recording settings for 10-bit color (limited to 10-bit 4:2:0 in their color formats).

While I have less than a few TB of storage on my working SATA SSD, I didn't want to fill it up so quickly by filming in ProRes even though it's an excellent codec to edit with. I didn't want to use H.264 mainly because it's a very old codec and there are other codecs that do a much better job at compressing data without losing too much quality (ex. HEVC and AV1).

After some testing and experimenting, I concluded that HEVC was able to achieve better image quality at smaller file sizes than H.264 (it's meant to be the successor anyway). AV1 isn't supported in Premiere yet so I couldn't use that codec, unfortunately. I have tried ProRes and the quality is similar to HEVC at the end of the day...given the kind of footage being captured (if it were live-action, then it may be a different story).

I still wanted great color quality so I chose to go with 4:4:4 chroma subsampling so that I get lossless color for postproduction. Thankfully, OBS Studio allows one to record in the I444 color format, which makes this possible.

As far as bitrate goes, I use a feature in OBS called CQP which is a variable bitrate control that works very similar to how the Constant Quality feature in Blackmagic RAW works. The bitrate changes to the best value for what's happening in the frame based on the CQ Level (which acts like a threshold, basically).

As a result, recording in HEVC 8-bit 4:4:4 gave me some of the best footage I needed to capture without compromising too much on quality and storage.

But yeah! That has mainly been my reason.

As I'm writing this, I'm slowly starting to realize that I probably should just get more storage. A NAS is on my list but a little out of reach at the moment.

I'm someone who loves the speed of SSDs but can be a bit ambitious on storage and would prefer to have internal SSDs than external because I'm trying to avoid creating a pile of filled up storage drives.

But hopefully, this helps give some more context to why I very much suggest for more hardware decoding support.

Thank you so much for pitching in!

Report · Dec 03, 2023

@RjL190365

I'm sorry, now quite a lot of editors are not dealing with 20 years of 4:2:0 decoding support. This is generally a HOME VIDEO. It's stupid to hear that. I do not mount materials from 4:2:0 at all due to the inability to perform color correction quality. Except for phone formats. And then with the iPhone 15 Pro/Max, now there is such an opportunity to record high-quality material in skillful hands. It is horrifying to hear that it is not possible to implement decoding support for basic workflows, at least 4:2:2. What will be discussed next is not difficult to guess. Now more and more editors are using 10-bit 4:2:2 material, and another 4:4:4 for high-budget projects. If we look at things and do nothing in this direction, then I'm sorry, what kind of performance will we be talking about, which everyone here has been buzzing about. We need to develop technologies and move with the times, and not stand and say that there will never be support. Are there engineers here and why is it not possible to work together with hardware manufacturers to do the "impossible" and bring the desired codec to hardware encoding? People have some of the top-end video cards that they write about here, but they do not enjoy their work. People spend a lot of money for what? Then why are video card manufacturers rushing forward so rapidly and more and more often with each new model they bring reality closer with editing programs, but in truth this reality does not exist. This is a myth. In simple words, if some people stand and watch how everything goes on, and others develop, then there will be no WOW effect and all this is an illusion.

Thanks

Report · Dec 03, 2023

I'm sorry to state this, but this decision not to support 4:2:2 is the decision of both AMD and Nvidia themselves. Adobe and all other NLE makers are left holding the bag, in this case. Even DaVinci Resolve does not support 4:2:2 hardware decoding at all with anything other than an Intel GPU because Blackmagic cannot circumvent the dictatorial policies of both GPU makers.

Report · Dec 07, 2023

I am very upset. Sorry, sorry, sorry. But is there really no solution and why are Nvidia and AMD developers not implementing the most popular 4:2:2 for today? I assure you it will remain the most in demand for many more years.

Report · Dec 07, 2023

Hey, Baffy!

Yea, that choice by both Nvidia and AMD to not support 422 is both odd and absolutely infuriating.

At least, with Nvidia, they could claim that since Intel can do this on the mobo, maybe they don't have to. But a lot of us run AMD CPUs and Nvidia GPUs.

For AMD ... I mean, really, what the hay?

Report · Dec 08, 2023

@R Neil Haugen I'm glad to hear from you, my friend.

Yes, I would really like to have such support on board NVIDIA. I haven't updated my processor yet, but Jensen Huang donated a lot of money to purchase the RTX 4090. You can congratulate me))) But, I would experience more joy with 4:2:2 support on a video card than on a processor. On the contrary, I believe that NVidia should have such support, despite the fact that the latest generation of processors support such performance, so that the user should choose in priority and capabilities what works best for him or what he currently has, the old i9900K processor (by the way, I have one on board now) or use the second tool is the RTX 4090. I think this is indicative and it would be right to give the user a choice depending on his capabilities. As for AMD, I can't say anything, since I don't use it.

Report · Dec 08, 2023

Agreed, I would want Nvidia to up their format support especially long-GOP.

But ... there's such a mish-mash between AMD and Nvidia and now ARC ... each has tempting capabilities for this and that, but not this other thing. I would, as a user, want them all to support a wider range of user activity.

Report · Dec 08, 2023

Have you done a comparison of recording HEVC 4:2:0 versus 4:4:4 for your workflow? If you don't need to color correct (tune your image in the software before recording it, that is one of the benefits of virtual production) and your output is 4:2:0 (anything going to the web) then maybe HEVC 4:2:0 might be ideal for your unique workflow. And it would be accelerated in PPro. Or you could proxy to 4:2:0 if you want to improve performance, the files are small.

Report · Dec 08, 2023

@Mike McCarthy hi.

Try to color the material 4:2:0 and 4:2:2. I'm not talking about 4:4:4. And see the result. I think you'll understand what I'm talking about here. The quality of work with the source and its flexibility in postprocessing are important to me.

Report · Dec 08, 2023

@Mike McCarthy I have, actually! I've worked on projects, recording in both HEVC 4:4:4 and HEVC 4:2:0 in 8-bit MOV and MP4. I tend to interchange between these two depending on the project.

Thankfully, the quality difference on my end isn't super substantial but that's mainly because I don't push my color grades that much, knowing the limits of the 8-bit and chroma subsampling.

In the Unity engine, I do record with the image having an HLG-like color profile to preserve some of the detail in the shadows and highlights. This does, in fact, help a good bit when in the postproduction phase.

Even though my case is centered around a virtual production, I feel the same should be considered for live-action productions as most creative professionals are recording in at least 10-bit 4:2:2 or higher.

If I do push the color grades for HEVC 4:2:0, then that's when I tend to notice the blocky artefacting and noise in the colors.

With HEVC 4:4:4 footage, I am able to push the grade more without introducing artefacting and noise by a modest amount.

So overall, there is a quality difference that you can notice if you really look for it and push it. Since this is CG footage, perhaps it may be more forgiving on the quality side of things when recording to a compressed codec.

In OBS, unfortunately there aren't any options that allow for 4:2:2 that's possible at the moment (there is the P216 color format, but that is 16-bit 4:2:2 and there is no way to record that with H.264 or HEVC at the moment).

Interestingly enough, the same playback issue does happen with HEVC 4:2:0. I talk more in detail about it in the post mentioned above but I'll put it here for convenience (I have it marked as the correct answer.):

https://community.adobe.com/t5/premiere-pro-bugs/hevc-8-bit-4-4-4-footage-is-very-hard-to-playback-i...

Long story short, I don't really have a good answer as to why it also happens with HEVC 4:2:0 other than that maybe Microsoft's HEVC extension is buggy.

I was trying to avoid H.264 due to quality concerns, but worst case scenario, I may have to suck it up and use that codec if it really comes down to it and use a high enough bitrate to still get a clean image.

Recording in 4:4:4 has been mainly to preserve as much detail as I can when in the posproduction phase before outputting as 4:2:0. For me, it gives a bit more flexibility; but I do sometimes use 4:2:0 for quick turnaround projects.

I guess what I could do instead is proxy to H.264 instead of ProRes so that I save a bit more on storage with the added benefit of hardware acceleration for H.264 in 4:2:0. That's a possibility.

But basically, I have tested and compared between 4:4:4 and 4:2:0 for HEVC.

Thanks for pitching in!

Report · Jul 02, 2024

Hi everyone! Gonna revive the thread over here with a little update I noticed.

In the Premiere Pro Beta v24.6, I saw this change regarding more hardware acceleration support for H.264 and HEVC media files.

If this is what I believe it is, then Premiere Pro may possibly now have hardware acceleration for 8-bit 422 and 10-bit 420 footage in the H.264 and HEVC codecs. That means much smoother playback!

The only part that kind of throws me off is the word "streams". I hope I'm not misinterpreting anything but feel free to clarify if I am.

In other words, since this post I did learn more about what works best for editing in Premiere when recording in H.264 with OBS Studio.

I learned that the NV12 color format is hardware accelerated so that's what allows Premiere to play back H.264 files a lot more smoothly than other footage recorded in H.264 (playback quality is actually quite similar to ProRes playback).

Overall though, I'm curious to hear other people's takes on this update change in the Premiere Pro Beta v24.6. Feel free to share!

Report · Jul 17, 2024

Adobe cannot overcome HW limitations. Both Nvidia and AMD do not support any 422 encoding/decoding and the support Premiere will have for 422 is for Apple's M chip based machines and for Intel's QuickSync.

Why doesn't Nvidia or AMD supports 422? it's not a HW issue, IIRC it's more of "blame the MPEG LA greedy requirements" issue..

Report · Aug 12, 2024

Recently, I have made research on this topic and went already through the same links and resources here posted/ mentioned.

Firstly, I strongly support the point made by @Eccentric Locust , and @Baffy19 . I am using Sony A 7IV, in which we have

XAVC HS (H.265) - Highly compressed

XAVC S (H.264) - Medium compressed

XAVC S-I (H.264) - Mostly un compressed

options for 4K videos. The chroma subsampling choices varies with different fps and bit depths (8 or 10).

I would love to edit with uncompressed XAVC S-I (H.264), 60fps where I am only opt to 10-4:2:2 depth and chroma subsampling.

Due to 10 - 4:2:2 this is not possible in Adobe premier, unless you have intel iCPUs which is capable of it. I DO NOT WANT TO UPGRADE, only for this reason, I am asking why it is not possible for GPU manufacture can not support the latest and best bit depths and chroma subsampling??

I am greatly disappointed, that I have to make proxies for my work otherwise have to shoot, 4:2:0.

The thing I do not understand is, the highly compressed codecs (i.e. XAVC HS (H.265) ) usually suppose to take CPU rather GPU according to the difficulties in the post describing. But adobe support the H.265, 10-4:2:0 even 60fps, for GPU acceleration.

@Adobe @NVIDIA @AMD Please take this topic as one of the feature request for future software - driver updates!

Being, a Software developer, I do not see the difficulties writing the codecs for other various bit depths and chroma subsampling, if you already have one or more working/ supporting format, that already released!

Report · Aug 13, 2024

Hi.
You don't have to go far here. Everyone who works or is related to color knows and understands perfectly well that it is impossible to paint good frames in 8-bit color. There is a severe destruction of pixels when creating a certain concept of a cinematic look. Just look at how many colors are contained in an 8-bit color. That's right, 16.78 million in color. And how much is in 10-bit? Billions already. To be more precise, 1.07 billion. I think it's not even worth persisting and squeezing out 8-bit sources to achieve the desired result. Modern cameras shoot in both 12 and 16 bit color, and just imagine what a difference it will be with the previous ones (12-bit = 68.68 billion!). If anyone here says that I paint and 8-bit is enough for me, for God's sake. But not for everyone, including me. The density of color in the modern world is important for flawless work and creating an amazing product that is pleasant to look at.
Good luck to everyone.