Using heavily-compressed H.264 for proxies

[Horshack]

I know the prevailing wisdom is to use a lightly-compressed intermediate codec for proxies like CineForm to maximize responsive editing performance in Premiere. However I'm looking for a more compact codec for a future 4K project that I plan to edit remotely on a notebook - a codec small enough where I can fit everything on an internal SSD. So to start I went to the extreme and produced an ingest profile that goes all the way down to 480P 1Mbps on H.264 using a 10-second GOP. That yielded a proxy that's about 1.8% the size of the original 4K H.264 (ex: 611MB -> 11MB for a 90-second clip). Based on some cursory testing (scrubbing, edits, various effects) I'm not noticing a performance difference between this super-compressed proxy and CineForm, even for large files. I'm using a quad-core i7-4770 to test.

Does anyone have advice on what specifically I should test with respect to proxy performance to make sure I'm not missing something that will bite me later?

[excited_Genie16B8]

Simple scrubbing shows a marked difference for me.

[Horshack]

Jim_Simon​, the only difference I see in scrubbing with the proxy is a long delay when I attempt to play backwards via JKL and stuttered reverse playback, which I don't do very often in my workflow. Scrubbing backward with the playhead seems responsive.

[excited_Genie16B8]

Well, if it works...

[Horshack]

Yep, seems to be working but I don't want to overlook something that will later kill my productivity. Like some basic edit function or effect sequence that is 10x slower with the compressed proxy.

[excited_Genie16B8]

I don't want to overlook something that will later kill my productivity. Like [an] effect sequence that is 10x slower

That could easily happen.  I definitely notice a marked difference in performance between Cineform and H.264 using only Lumetri.

I've said this before and I think it bears repeating.  Editing requires LOTS of hard drive space.  It's always better to accommodate that than to try and work around it.

[Meg The Dog]

Any time you are using an interframe codec such as H.264 versus an intraframe codec such as Cineform or ProRes you are adding an additional workload to your computer.

For example in a 24fps project, Cineform presents 24 unique frames to the computer per second, while H.264 presents 3 packets of 8 frames each. The frames inside the packets are mixed together like pieces of a jigsaw puzzle in a carton, designed to minimize the amount of space the puzzle is taking up while in storage - but needing to be reassembled prior to being seen as an actual image.

To play the Cineform the computer just plays the frames one after another, to play the H.264, the computer has load the GOP, reconstruct the individual frames out of each packet, and then display them - continuously.

The apparent operational consequences of choosing one over the other for editing (as opposed to the impact on image quality, recompression artifacts, etc.) are basically a matter of computing horsepower - you need and will use a lot more horsepower to work with interframe codecs.

MtD

[Horshack]

Yep, understood about the computing implications of using a long GOP - in fact I'm using a 10-second GOP for this test ingest profile. I'll have access to AC on the notebook so power consumption from a higher CPU utilization wont be an issue.

[Horshack]

FYI, I just compared CPU utilization between my 1Mbps H.264 proxy and CineForm and they were identical for forward playback - 20%. For reverse playback the CineForm stayed at 20% while the H.264 went up to 80% and was choppy. So for those who don't do much reverse playback perhaps it's time to reconsider the prevailing wisdom against H.264 as a good proxy choice, esp in light of all the HW/CPU acceleration currently available for H.264 decoding. For a 1:45 4K clip, CineForm proxy was 700MB vs 14MB for H.264.

[Horshack]

I reduced the GOP length to a more reasonable 1-second and now reverse playback plays with only a slight start delay and no choppiness. If anyone is interested in trying out these proxy presets I posted them here:

Horshack Premiere Pro H.264 Ingest Presets

There are four presets:

• 480P, 23.97fps, 1Mbps
• 480P, 29.97fps, 1Mbps
• 720P, 23.97fps, 2Mbps
• 720P, 29.97fps, 2Mbps

The 1Mbps preset reduces the size of my 4K H.264 files by slightly more than 98%.

To add a preset to premiere:

1. Open up the Media Browser (Window -> Media Browser)
2. Click the wrench next to the "Ingest" checkbox
3. Go to the "Ingest Settings" tab
4. Click the "Ingest" checkbox, change "Copy" to "Create Proxies"
5. Click "Add Ingest Preset" and specify one of the four preset files I shared

[chrisw44157881]

there's also dnxhd LB small and prores proxy LT and cineform small bitrate which are all wavelet.

in Mb/sec

Apple ProRes (VBR): Proxy - 36

Avid DNxHD (CBR):DNxHD 36 - 36

GoPro Cineform (VBR):Low (1) - 64

https://forums.adobe.com/thread/2311148

Also, it's the actual h.264 long GOP IPB frames that have to be reconstructed. h.264 has a bad reputation for slow cpu from IPB frames. In actuality, h.264 can be very fast with all I frames, or even no compression at all!!

[Horshack]

I think H.264's bad reputation is a hold-over from when decoding was all software based without any CPU/HW assist. That's no longer the case. Certainly there will still be cases where GOP decoding is noticeably slower (like reverse playback) but for general editing I still haven't found any gotchas.

[R Neil Haugen]

I've seen some long and detailed pieces by those who build high-end editing & colorist rigs explaining why long-GOP media is a killer even on their rigs ... and why they do NOT recommend using that media for major editing, VFX, or colorist work.

Your mileage will of course vary ... but as always, test first.

Neil

[Horshack]

I agree, I would never grade or even perform basic exposure adjustment on proxies. I use proxies only to edit.

[excited_Genie16B8]

I would never grade or even perform basic exposure adjustment on proxies.

I actually find the Cineform proxies of sufficient quality to use for CC and grading.  At least for my meager work.  (Corporate/Events)

[excited_Genie16B8]

I think H.264's bad reputation is a hold-over from when decoding was all software based without any CPU/HW assist.

No, it still sucks.  At least in comparison to options like Cineform and DNx.

[nicklear]

I have had good success with using these presets on my system for H.264 proxies. Where my machine struggles playing back 4K H.264 off my RAID, it does very well with the lower res 720p H.264 proxy files which I can fit on my SSD drive. For me if it works, it works.

The main problem I having is that Premiere wants to make a .cfa audio file for each proxy file, as it does with any file that isn't 48kHz uncompressed audio. Is this normal? Do the cineform and prores proxy encoding settings use wav audio?