Encoding video to HEVC H.265 with alpha channel - can't get transparency

[KrispyKreative]

I'm trying to convert ProRes 4444 with alpha to a HEVC H265 with Alpha. I'm on an M1 Mac Studio (running Ventura 13.2.1). I've seen people say "hardware encoding" needs to be selected in Media Encoder (23.2.1), but it is not selectable. I've tried this conversion using Premiere and in also Mac finder using "Encode Selected Files"; everything seems to work correctly, but when I test the finished HEVC in AE or Pr, they never have alpha channels. Just black where there should be transparency. What am I doing wrong? Thanks.

[Warren Heaton]

The specification for ProRes supports an alpha channel while the specificaion for HEVC does not.  

When something goes against file format specifications, like having transparancy in a HEVC,  it is often referred to as "out of spec".

Howevever, the Apple AV Foundation Frameworks Team figured out a way to include transparency with HEVC even though it's out of spec.

Right-click on an Apple ProRes 4444 with alpha file in the Finder and choose Encode Selected Video Files.  When the Encode Media dialog box appears choose one of the HEVC options from the Settings pop-up menu and enable "Preserve Transparency".  Then click Continue.

macOS Finder Encode Media dialog box set to HEVC with Preserve Transparency enabled

The resulting file will be HEVC H265 with alpha.

Since this is out of spec, be sure to double-check that the application that the resulting file will be used in recognizes the transparancy as expected.

[KrispyKreative]

Thank you. When I do this exactly as outlined, it appears to process correctly, but when I bring the finished file into Premier or After Effects to check the transparency, it appears black rather than transparent. Any chance it actually has the transparency, but Pr and Ae just can't see it?

[Warren Heaton]

@KrispyKreative 

Yes, as HEVC H265 with transparency is out of spec for HEVC, interpeting the transparency in that format is not supported in After Effects or Premiere Pro.

After Effects and Premiere Pro need the alpha to be Straight or Premulitiplied in a file format that supports it (as do many other applications).  

If HEVC H265 must be used (keep in mind that it's a delivery format), transparency can be included by pairing up an RGB file with and Matte file (also called "Alpha Only").  If doing this, it's common to name the RGB pass "fileneame.mov" and the corresponding Alpha Only pass "filename_matte.mov".  In tbis case with HEVC, it could also be filename.mp4" and "filename_matte.mp4".  In the days of analog video, this was commonly called a "hold out matte". 

[3dron]

Can this be done in the Windows version of Adobe Media Encoder (I've tried 2022 and 2023?)  I cannot find where to right click on "Apple ProRes 4444" and get "Encode Selected Video Files"  when you say in the finder where should that be in Windows?  In Windows the only place I can right click on "Apple ProRes 4444" is in the Queue and only get Export Settings, Skip Selection, Duplicate and Remove.

[Warren Heaton]

As far as I am aware, having transparency with h265 encoding is a feature exclusive to AV Foundation Frameworks.  As such, it's limited to macOS.

[Synyster27062305aswn]

if the data is in the container is there no way to seperate the channels, data in whatever format is data, i find it wierd as i can write cuda to process any matrix or set of matricies in anyway i choose to write. i find it strange that channel splitting isnt an option in adobe software, rgba is rgba regardless, if the data is there, the issue is the reader of the data not the writer. back to blenders vfx editor and good'ol python for me 🙂 et 'el hard-mode 😛