Project Manager / Render & Replace are missing an ESSENTIAL feature

Report · May 18, 2021

Hello there,

Right, where do I begin with this one...

Firstly, I'll state that I prefere to use Render & Replace rather than the Project Manager, as it's simplicity and functionality is better, and the results are exactly the same. I got used to using Render & Replace because the PM used to have issues when using Consolidate and Transcode with Handles... Basically, it didn't work if your source media is an Interframe codec, and would copy the whole source media file rather than trimming it. It's fine with Intraframe codecs, for the obvious reasons - the project manager could not process/break the Long GOP structure of Interframe Codecs.

I'm not 100% certain if the Project Manager has been fixed for that issue, as I've stuck with Render & Replace which could always consolidate Interframe codecs. Why Render & Replace can do what the Project Manager should be able to do is beyond me, but hey ho, there we are.

Okay, so, the problem...

I have created a super simple timeline to demonstrate the issue. I've used a PiP moment to make it clear; in this screen grab you can see a BG clip, and two resized clips creating the PiP:

All three of the clips come from the same source media file, you can see the name on the clips on the timeline.

[img]https://i.postimg.cc/N0fb7z9h/Image3.png[/img]

I DO have Show Duplicate Frame Markers enabled, but as these clips are from different timecodes in the master source file, obviously the dup frame markers are not showing anything.

Now, I want to consolidate the media, so each clip on the timeline has a VERY specific number of handles and the clip is 100% independent; this is for addition post-production pipelines in VFX and grade.

However, when I go through the consolidation process, be it Project Manager or Render & Replace, if the clips on the timeline do share any frames, then the consolidated media files are merged into 1 file. This is bad. The transcoded media files should be absolutely unique and not shared with any other clips on the timeline, regardless of shared frames.

This is my new consolidated section of the timeline, as you can see, it all looks okay -

[img]https://i.postimg.cc/DzrJ0vyn/Image4.png[/img]

However, when I drag out the handles, you can see that the 25 frames handles are not there, and the Show Duplicate Frame Markers has kicked in, revealing that the 3 clips on the timeline do in fact share the same, newly transcoded, source media file-

[img]https://i.postimg.cc/7h2yP6wN/Image5.png[/img]

I need the consolidation to be absolute, like I said before, with each clip on the timeline having the same 25 frame handles. There can be no variations on this.

I did a quick work-around to demonstrate what the final result should be, but this involved doing each clip individually, which is obviously not practical in real world work environments, especially if your duplicated frames are hidden in handles. One can't be expected to drag out handles of every clip to see if there's any dupes. That would be insane. You can see in this image that I've dragged out the handles to their maximum, and each clip is exactly 25 frame handles, and you can also see that the name of each clip is different, with each clip having a unique source media file. This is what we need to have the option to create when using Project Manager and/or Render and Replace.

[img]https://i.postimg.cc/vTH3X2Sx/Image6.png[/img]

Please can somebody enter this discussion and take this forward. I'm curios to find out if there are any time-efficient work-arounds for this, but I'm pretty experienced at all this stuff, and have explored every possible option and setting in Premiere, and have reached the conclusion that Premiere does not have this functionality.

I look forward to any response.

Thanks.

[Moderator note: I added your images into the post.]

Report · May 18, 2021

Damn it. My screen grabs are not showing, and there doesn't appear to be an option to edit the post!

Report · May 18, 2021

Please post screenshots here using the forum so we dont have to download unknown files.

Report · May 18, 2021

Yes, I tried that but it didn't work for me. It kept stating I should fix highlighted errors, but there was nothing highlighted. Total size of all images is ~850KB, and there's only 6 of them.

Report · May 18, 2021

I've just copied/pasted the first post into a new post, with the screen grabs uploaded using the Adobe website protocol, but it's not working. I'd love to be able to share my screen grabs with you, but Adobe doesn't want to.

Report · May 18, 2021

I guess you can click on the URLs in the OP to open them up on the image hosting website. Better than nothing.

Report · May 19, 2021

Thank you Moderator, for adding my pictures.

Report · May 19, 2021

Hello,

I wonder what is your workflow with post production for needing all clips to be independant and have exactly 25 frames handles ? I don't understand why it should be needed.

Also, do you really need to have your files transcoded ? (See below why I'm asking this)

I don't know any native ways to trim the files without re-encoding, even with Intraframe Codecs. You said it was possible at the beginning of your message, are you sure of this ?

It is not a solution for your issue, but it could become one : PlumePack. It's a third party paying plugin which improves the native Project Manager. Its special feature is to trim the files without re-encoding. It works with Interframe Codecs (H264, H265), by trimming between GOPs so it doesn't break them.

In the next release, PlumePack should be able to trim files by parts (For now one media Source file -> one Trimmed file from Min/Max use in timeline). However it will do it in a smart way (if 2 parts shared some media frames, it will export only one part) because the main purpose is to optimize disk space.

From your need and if you don't need to transcode your file, we could add a parameter to create one trimmed file per "part" in timeline. However for interframe codecs if you specify 25 frames for handles, there could be more handles (in order to not break the GOP !). What do you think ?

More infos on PlumePack : Autokroma

Best,

Nicolas from Autokroma

[edited by mod: spam]

Report · May 19, 2021

Hi Nicolas,

Thanks for your input. When you say - "I don't know any native ways to trim the files without re-encoding, even with Intraframe Codecs. You said it was possible at the beginning of your message, are you sure of this ?" I didn't write that, or i've written it in such a way that is misleading. There is no way to trim the files without re-encoding.

In regard to my desire to have independent media files, with 25 handle frames, it's to do with farming out individual shots to Nuke and Houdini artists. Actually, the number of frames in the handles does not have to be 25, we usually work to 10, but it's the principal of being able to specify and know exactly what the media is.

It's a real pain to have to sift through consolidated media and do an additional phase of work, and a lot of it, when handing out shots to Nuke/Houdini artists if the shot they're meant to be working on is part of a larger media file because handles overlapped. I'd like to be able to efficiently hand over a shot, and the artist knows immediately what they have to work on, without any margin for error.

Also, there is the issue of the time it takes to cache files in Nuke and Houdini, and if every artist has exactly what they require, it doesn't waste time cacheing frames that are not required.

There is also the additional phase of conform and online to consider - if the shots that Nuke/Houdini artists render out are exactly the same number of frames as in the consolidated edit, it makes this process infinitely smoother, by simply being able to unlink / relink to the new shots. If the artists have to render out a long clip, but their portion of the work is a small part of that clip, it's a huge waste of time.

Generally, it's about being precise, and knowing exactly what we've got, and reducing any chance of errors in the conform.

Report · May 19, 2021

Hello again,

Just to add, I want to farm shots out to various Nuke/Houdini artists all working remotely. There can be no room for error.

Report · May 19, 2021

Hi again Nicolas,

If your company can create a third party plugin that could achieve what I require, I'd buy it immediately, and recommend it to the many people I know who deal with this problem.

It would have to work with Interframe and Intraframe, from h264 through ProRes4444 and up to TIFF /DNG / RAW image sequences.

That would be amazing.

Report · May 19, 2021

We should be able to add a feature to PlumePack which would do what you want. From what I undestood, it seems to be able to trim your files (H264, ProRes, Image Sequence, ..) per clip in timeline, without having 2 clips with the same Media Source.

I didn't well understood why having 2 clips from the same source is a problem though. In my opinion, let's say you have 2 parts of the same footage in 2 clips. If trimming / transcoding created one final clip with both parts inside (and no more frames), then it should be quicker to send to your colorist / VFX artist, and also it would be faster (the colorist / VFX artist would need to use only one footage instead of 2). I don't see why relinking would be harder in that case ? You would just need to replace the project item and that's all !

The feature I'm talking about here in PlumePack should be released quite soon (we hope to finish it in June, so let's say July max). We didn't think to add a parameter to do what I said here initially (I mean : having all clips with different sources => It's not really optimizing the project so), but we could add it if you need it.

Also, if you have troubles while relinking your footage in PremierePro after sending it to the colorist / VFX guy, we would love to more understand why and see how we could help too.

Please contact us at plumepack[at]autokroma.com (replace [at] by @) it would be easier !

Best,

Nicolas from Autokroma

Report · May 19, 2021

I'll try and explain a typical scenario... Say we have a consolidated media source file, that is a combination of 2 or 3 timeline clips merged into 1, and is a duration of 400 frames, and this clip goes to a Nuke artist who is only working on 100 frames, and the clip is rendered out at 100 frames + 10 frame handles = 120 frames.

Now, when the 120 frame render is conformed/onlined in Premiere, that is expecting a 400 frame media file, it all goes out of whack, and additional work is required to conform. Now imagine this simple scenario multiplied by 500, with clips coming in from different countries, believe me, it's not a pleaseureable task.

If the Consolidate and Transcode could simply make each clip, and corresponding media file, absolutely unique, with a user specified handle duration, it would be a game changer.

Also, it's not about optimizing the project based on storage capacity/limitations, it's about opimizing the project for smoother VFX pipelines.

If you could create such a feature, I would promote it, and recommend it to all my colleagues and extended network.

Report · May 19, 2021

Also, a transcode feature would be ideal, not just trimming, as we like to ensure that all media files are Intraframe when they go to artists.

Report · May 19, 2021

Thanks for the feedback I see why now.

I think we will add this for the next release. The transcode feature is also one of the features we would want to add in the future to PlumePack, but it will not be for the next release though.

Please send us a mail to say hello, so we will send you a Beta as soon as the feature is ready to test, to be sure it is ok in your workflow !

Best,

Nicolas from Autokroma

Report · May 19, 2021

Hi Nicolas,

Yes, I will do that, I will send an email.

Very exciting!

S.

Report · May 19, 2021

Good morning,

I've done another test this morning to see if the Project Manager can actually consolidate Interframe codecs now (it's been such a long time since I used it), and amazingly, it does!

So, I went ahead to see if it can do what is says it can do, ie - individual clips. For the record, here's an Adobe tutorial on consolidating to Individual Clips.

https://www.youtube.com/watch?v=huyEKvXEeIw

And here's my experience based on my requirements for a top-level professional post-production/VFX environment -

To make it simpler this time, I simply stacked the same clip for the PiP, which you can see from the Show Duplicate Frame Markers. I didn't bother with the handles thing because, well, let's cut to the point - if it can work with direct dupes it can work with handles.

Here's my PiP and my 'locked edit' pre-consolidated timeline, showing the dupe frame clips.

I got to the Project Manager, and set it up in the correct way -

Now, from this setting, I would expect the resulting Project that is created from this setting to make each clip on the timeline to have it's own, newly created, independent source media file, ie - the source media file is duplicated. Each clip on the timeline will also have 20 frame handles.

After the Consolidate and Transcode process I open the new project and the result is wrong, or maybe not 'wrong' but it certainly is not suitable for a sleak, efficient, VFX pipeline, or even for prepping for a Grade. This is not good.

This is the resulting timeline in my new project -

You can see that the 3 stacked clips are not independent, they share the same source media file, as revealed by the presence of the Show Duplicate Frame Markers.

I have dragged out a couple of the clips to the full extent of their handles, made an edit, and coloured the handles in Rose. So, basically, the Rose coloured bits are the handles of the clip.

You can see the 3 stacked clips do have the correct 20 frame handles, but that's only because each clip is a direct dupe so there's no variation. However, the other little clip to the right highlights the other problem - the left side handle of that clip is much longer because another clip on the timeline (in this case it's the clip immediately before, but it could be anywhere, same problem) shared some frames in their handles, so premiere decides to merge the resulting Consolidated/Transcoded clips together. You can see that the red Show Duplicated Frame Markers reveals that the 2 clips share the same source media file. This is not good.

It would be absolutely amazing, and SO EFFICIENT, if there was a simple check box that overrides this behaviour. Each clip on the timeline should be 100% absolutely independent, and not share the same source media file, regardless of dupes, etc.

Please discuss. If anyone has solved this in a practical way, I'd love to hear how...

Thanks.

Report · May 19, 2021

I'd like to be able to change the title to be a Feature Request.