Transcoding Red weapon 8k raw files to pro res, best settings for edit workflow?

[jpdirector]

Hi, I am in uncharted territory on many fronts here and am looking for any suggestions that may optimize workflow. I am about to begin a new edit on a feature film in Premiere Pro cc 2018, with a brand new iMac pro, (10 core, 128gigs, Vega 64) which I hope will give me the ability to edit without creating proxies, (I am aware that editing the Raw footage in real time is not realistic) The entire film, (other than some 4k drone footage) was shot entirely in 8k raw footage, and so I have planned all along to have to transcode the footage to Apple Pro res (422?)... Because both the iMac Pro and Red 8k tools are relatively new and it doesn't seem many people have used them together yet, I can't find a lot of tutorials or information on the best settings to optimize workflow in our initial set up for ease of use on the back end. Obviously, we could do it the "old" way and create proxies, but our hope was that with this new iMac Pro that would not be necessary. Any opinions, information or ideas on this would be greatly appreciated. Transcoding vs. Proxies has been a topic that has confused me for some time, so again, any insight is helpful. Thank you!

[R Neil Haugen]

Serious questions there ... maybe Jarle Leirpoll​, Kevin-Monahan​, or SAFEHARBOR11​ could help with this one.

Neil

[jasontcox]

Well, it's true, since the iMac Pro is SO new, it's hard to say how Premiere would handle 8K raw... there have been demos of it working great in FCPX but Premiere probably isn't fine tuned for it yet so my best guess is you're PROBABLY still going to want to/need to make proxies. It's actually a pretty easy workflow. A lot of the info you need is here:

Adobe Premiere Pro Help | Ingest and Proxy Workflow in Premiere Pro CC 2015.3

Don't worry that it says 2015.3, the settings haven't really changed. In the Ingest Settings window (when setting up a project file) you'll want to select Copy and Create Proxies and then choose a Format like ProRes and a Preset that is probably like 1/4 the resolution of your 8K resolution. You can make your own presets in Media Encoder and import that as well if there isn't a perfect preset. Then when you import media, it will duplicate the raw files to whatever location you choose in the Ingest Settings window AND create lower res proxy files. There is a button you can bring up in the button bars of the Source and Program Monitors that will then toggle back and forth between your proxy copies and your raw copies. Spend 90% of the time with the Proxies active to make Premiere run MUCH smoother and just toggle to the RAW files when you maybe want to do accurate color grading.

Hope that made sense! Let me know if it didn't.

[jpdirector]

Thank you! Yes, this all makes sense and we have seen some recent tutorials using a similar workflow. I will check out the link to do some comparisons. Even when using this approach there are still so many sittings to make decisions on. Still a little confused as to why there are both "create Proxies" and "Transcode" options? In the end, we're trying to edit with the highest quality video in the original timeline to optimize a quick workflow and to give us a better idea of which shots and takes are visually the best, (before any grading, effects or color correction... I guess that's what we all want). Holding out hope that we can playback in our timeline at full quality in an apple pro res format without issues. So, would we consider an apple pro res 422 or 444 a "proxie" coming from 8k raw? Thanks again!

[jpdirector]

Hi Jason, I have a follow up question that I may post as a new discussion topic... Do you have any idea what the conversion ratio is for file size of the proxy files? I understand that setting choices will make a big difference, but in general our concern in setting up to edit this feature is what size G-Raid drives to get in order to house all the Raw media files, (15T total) as well as the newly created proxy files which will be linked to them. Ultimately, the consensus is that we keep them all together for smooth workflow near the end of the project when we are getting ready to start doing outputs. I've read on some blogs and forums that the proxy files end up being as much as 1/2 or 3/4 the size of the Raw files. That means we need 32T? of space just for the media of this movie?

Thank you!

[Jarle Leirpoll]

This chart may help answer your file size question: https://blog.frame.io/2017/02/13/50-intermediate-codecs-compared/

[jasontcox]

Sorry for the delay in getting back to you. Jarle's link is probably exactly what you need to work it out! A half-res (4K) or quarter-res (HD) proxy file should require a FRACTION of the file space compared to their 8K RED raw counterparts. I'd probably go with the "ProRes 422 LT (HD)" (or standard ProRes if you want slightly higher quality proxies) setting to edit and just switch to the RED RAW when you need to color grade.

[jpdirector]

Thanks Jason, Yes, that is a great suggestion and the one we have been doing tests in. Our new issue is that there are so many sequence presets for Red R3D files now in Premiere CC 2018 and we can not find a clear answer on which one is "best" for workflow. We will ultimately be mastering and outputting to 4k and want to minimize any issues now in the sequence settings for when we output. Any experience, opinions or advice on this would be greatly appreciated. Because it is a feature, there will be many different sequences I'll use while editing and sorting through footage and every time we create a new one we have to choose the sequence settings. Thanks again.

[jasontcox]

Im sure someone will have a difference of opinion or perspective here (chime in, folks!), and it's hard to cover ALL possible variables here but first and foremost, I'd stick with 8k resolution settings for the sequence rather than possibly selecting 4K. Processing wise, it's usually best to have a sequence that matches your clips' resolutions (and later simply choose 4K export settings when done). You're going to have to make your own Sequence preset likely. The only exception to that is if you thought you might want to zoom in on some of your clips often and take advantage of the extra resolution you shot with... then that is a situation where I'd pick a 4K resolution sequence.

Then you've also got to consider Video Preview settings and if they're super important to you. This is the codec used to create render files. There are two major schools of thought here.

1. The Preview File Format isn't important and simply produce fast, low quality preview files (especially if you drop their resolution). Export will render each and every frame from scratch based upon export settings.

2. The Preview File Format IS important and you very carefully pick a Preview File format that matches what you know will be your export format. This will sometimes slow your render file creation (compared to option 1) but allow you see higher quality render files AND to use your video preview files in your export process, speeding up the export.

[Guest]

You would probably be set with proxies encoded in ProRes 422 / HQ at either 1080p or 4K res. Either should play back fine so long as they are on a fast disk setup (either SSD or RAID) which I'm assuming you have with a mac that high end. If you need to pull those clips into After Effects the proxy is attached along with the original clip. It works really well. Once you've created and attached the proxy files, you can simply toggle them on and off with the press of the toggle button to gauge focus or mess with the R3D settings.

Adobe Premiere Pro Help | Ingest and Proxy Workflow in Premiere Pro CC 2015.3

EDIT: I see this answer was pretty much given above already! Apologies for the similar post. Good luck JP and let us know if you have any other questions!

[jpdirector]

Thank you, spnsir! Yes, we are using a 8T Raid drives, with thunderbolt ports, and this is very helpful, especially with the recommendation to use the ProRes 422 / HQ at either 1080p or 4K res for the proxies. This is one of the things I can not find consensus about. And, yes everyone here is right on when linking this question for the set up in workflow to the output. There will be some effects, color correction and grading done which we knew while we were shooting. So now, we just want to streamline the edit workflow as much as possible. I was a FC7 user, (and all previous versions for 18 years) and every time there was an upgrade or a big jump in computing power or cameras, (film, DV, DV cam, HD, 4k now 8k...) I hoped for an easier faster solution for the big feature film projects. And, ultimately outputs would become a stressful affair if there were mistakes made in the set up or any of the settings, (which always have so many options). Also, to answer your other question about my laptop, I'm actually on a brand new decked out 2018 iMac Pro now and have included the specs in the original post. That is partly what prompted this question as no one has experience with the new iMac working with the Red 8k RAW files yet and I think everyone is going to be curious what the machine is capable of. Thank you again!

[excited_Genie16B8]

I think you're better off using native Proxies over Transcodes.  The Proxy work flow will default to the original media for final export, and you retain access to the RAW controls.

Work offline using proxy media |

[jpdirector]

Thank you! That seems to be the consensus. That was my main question, how can I maximize ultimate output ease while at the same time streamlining the editing workflow with the power of the new iMac pro.

[Jarle Leirpoll]

I actually think you may be able to work directly with the 8k files!

RED RAW files are special in that they use Wavelet compression, which makes it really, really easy to read 1/2 res, 1/4 res, 1/8 res etc. The way wavelets work, you get more resolution the further you read into the file. So you get 1/16 res first, then 1/8, then 1/4, etc. See image here: Wavelet for multidimensional signals analysis - Wikipedia If you read all the way, you get full resolution. This means you don't have to read and decode the whole thing to show partial resolution.

Think of it as built-in proxies in the file!. To get access to these "proxies" in Premiere, just lower the playback resolution in the Source and Program Monitors. You can still watch full resolution while pausing, for quality control.

This is in stark contrast to the 4k drone footage you'll get, which is probably H.264, and will need much more computer power than the RAW material (at 1/8 res). With H.264, you need to read the whole thing, and then you can throw away pixels to get a lower resolution.

My laptop can edit 8k RAW from RED like butter when I use 1/8 res, but struggles with 4k drone footage. I'm not talking about just being able to play the stuff back. Editing involves a lot of scrubbing back and forth, too.

I have not cut a real project with 8k material, though, just tested a few shots, but I have edited 4.5k RAW projects in Premiere on the laptop, and it was all smooth riding. I have 3 SSD drives and a good GPU with 8GB Video RAM, and 32 GB RAM. The GPU is important here, since it's doing the de-Bayering of the RAW files, plus the scaling (down to 4k) and color grading etc. Fast storage is also important, these are large files.

It would be interesting to hear about other people's experience with RED RAW in Premiere, though.

[Guest]

Sidenote: what laptop model do you have and are the SSDs external or internal?

[Jarle Leirpoll]

My laptop is an HP ZBook 17. It's a powerhouse! All three SSDs are internal. There's normally room for two, but I opted away from the DVD drive to get room for one more. 🙂

[R Neil Haugen]

Just looking at a new laptop, great timing on the comment.

Neil

[jpdirector]

Wow! This is awesome information and news. I will try some tests just lowering the resolution for playback and editing. And, yes, I am going to be asking a LOT of the machine in that this is a feature film with all the trimmings, so as you know the edit will get very dense the further we move forward toward completion with all kinds of additional files added to the timeline, (effects, sound, score, color... ). I had no idea the Red Raw files worked that way with the Wavelet compression??? Great info. I am a writer/director/editor and while I have shot a lot of my own stuff, I'm definitely NOT a DP or camera expert. So, I find it really difficult to keep up with the developments on that end. Thank you again! I was told by both my Oakley (Red makers) contacts and my Mac buddy at their Apple business team that Red and Adobe are working very closely together, and that in fact Apple might start actually selling the Red cameras with support, specifically geared toward the Premiere crowd??? We'll see I guess. I'll let you know how this goes with the new iMac Pro. Thank you again for this education on wavelet compression.

[R Neil Haugen]

Jarle and spnsir​ and Jim_Simon​ ... thanks for popping in, it's great to get some good knowledge and information/suggestions for something like this!

Neil

[janm25320206]

Editing on 8K with quality on 1/8th is doable on current Mac hardware. However, the larger your project becomes the bigger the chance of problems. Even though the hardware "supports" it I've noticed a lot more unstable projects. 

If you want to quickly convert RED .R3D to ProRes I suggest: www.proxybeuker.com 

[R Neil Haugen]

They've got the announcement banners going on the new Productions option when they 'drop' the next version, I'm assuming next week sometime as that would have been NAB and they've got their schedule of live-streaming presentations starting Tuesday.

Productions looks to be a major game-changer, as ... it provides an overall structure for housing many projects inside the new Production folder in Premiere ... which links individual project files under-the-hood so it enables the use of bits from one project file in another one without the duplication of assets that happens now. Soule says they've been using this with all their Hollywood teams for a while, and breaking large projects done into a series of much smaller linked projects within a Production folder speeds everyone's computer.

I'm a solo shop, and ... that should allow me to simply put all my projects both for our own facility and external clients in one Production 'folder' ... and be able to search the whole shebang for b-roll, graphics, audio files and such ... nearly like a full DAM in some ways.

So ... splitting a large project with much hefty meta-driven RED/Arri/Sony-log/BMD-RAW media into scenes or 'acts' within a Production folder in Premiere would mean that loading any section to work would put vastly less stress on the computer. But the whole project is also 'there' in one place, easy to work on whatever you need. Access all media/files ... so for you, it should make working with those RED files so much easier.

My projects are relatively small, and occasionally involve multi-k RED files. I get through them pretty decent on a mid-level computer. Being as the project itself doesn't take nearly the overhead a massive single project file would. So my rig should be capable of actually working a much larger project using Productions organizational stuff.

Neil