Any interest in utility to prune source media files used by project?

[Horshack]

I'm guessing this is a common issue - you have a ton of source footage but after your edit is done you only used a small percentage of it. If you have no interest in preserving the unused source footage you can easily delete the files you haven't used. However this doesn't address the issue of footage files you used partially - for example if you have a 30 minute source file but only used 5 minutes of it.

One way to avoid this scenario is to do a rough cull of your files early in the edit process and resave the source segments you want to keep, but that requires re-encoding your sources. If your workflow already involves transcoding then this isn't an issue. However if you don't already transcode this means either a generational loss from recompression or using one of the third-party tools that allow cutting source media files without recompression (SolveigMM AVI Trimmer + MKV, Video Splitter 6, avidemux, etc..). The problem with these third-party tools is that you have to perform the rough edits/pruning in their tool, which generally aren't as fast and comfortable to use as performing the edits in your preferred NLE like Premiere.

An ideal solution would be to do your rough edit/pruning in Premiere and have a utility which can take your project file (or some export from it) and do the splitting/concatenation of your source files automatically, after which you could delete your original, larger sources. So far I haven't found any third-party utility which supports cutting based on the ingestion of a project/EDL file - if anyone knows of one please reply to this thread with the details.

An even better solution would be for the utility to not only perform cuts of your source files but also generate an updated project/XML file that updates your original edit list with references to the newly created split/concatenated files. This would allow you to take a completed Premiere edit and automatically prune it to the source material you used, without requiring a rough edit/prune at the start of the editing process.

Based on my own personal needs I wrote a quick proof-of-concept utility last night (written in Python). It ingests a Final Cut XML file (exported from a Premiere project) and for each cut/clip in the file, launches ffmpeg to perform a lossless demux (basically a cut) to extract the portion of the source media file into a new file (with support also to concatenate multiple subclips into a single file). It's not a frame-accurate cut since ffmpeg only supports copy-transcoding on i-frame boundaries, which means the subclips will be slightly larger than the actual cut in the project file (by up to the GOP length, which is typically 1/2 second). This isn't an issue for me since my requirements are only to do a pre-edit cull of my source files. However if there's enough interest I believe there are open-source alternatives to ffmpeg that support a smart-render around the edit point (they'll transcode the area around the i-frame boundaries).

How many of you would find such a utility useful? If there's a lot of interest I would consider making this a larger project so that others could use. It would be open-source and free.

[Jim_Simon]

I just don't like the idea of deleting original media.

Problem solved.

[Horshack]

You would still have your original footage - just pruned to the portions you actually used. That said, some prefer to preserve all their source footage (even the parts they didn't use), so they wouldn't find utility in this.

[landru515]

This sounds very similar to using the PPro Project Manager to consolidate and transcode the project down to just what you are using in the sequence. You could add massive handles if you wanted to so you'd have extra footage at the head and tail of the clips you're using in your cut. Is what you're proposing something different than this?

I do agree with Jim that permanently deleting original media isn't an option, at least for me. Storage space is cheap and you never know what you might need in the future, especially if lots of your projects have similar types of footage in them. Oftentimes though once we lock our edits we will go through this consolidate and transcode process with the Project Manager in PPro, then we will basically start off with fresh simplified projects where we can do our onlining/finishing. At that point we delete all of the transcoded proxy footage and other media off of our working storage space as we just need the consolidated (or copied in Project Manager) project.

[Horshack]

It's the same concept as the Project Manager except the archival is with the original source media rather than with transcoding. The issue with transcoding is that it's a grossly inefficient archival mechanism (at least if your source footage is typically H.264/H.265) - the storage space saved by pruning the project footage is lost to the larger bitrate required to maintain the quality of the footage.

[Tara T]

My personal need is to have something like MPEG Streamclip that trims w/o re-encoding so I can trim away what I don't want, but that works for 10-bit files.  When I use a clip in a timeline, I wouldn't have a need to delete what wasn't used for that project -- I would still want to retain the entire clip that I kept when I did my original cull.   I shoot wildlife, and frequently let a camera run on a woodpecker cavity/nest box for a couple of hours, and then scrub through and just save the parts where there's actually a bird present. 

[Horshack]

Hi Tara, this utility would accomplish that. I believe ffmpeg pretty much supports every media type, including 10-bit files. Since starting this thread I've been working on the utility in my free time - I should have a beta ready in about a week if you're interested.

[Tara T]

Sure, that sounds great, thanks.  I'll also check out ffmpeg.

[Horshack · Accepted solution]

I published the app today:

FYI: Precut, my open-source app for splitting media files without transcoding

[Szalam]

This looks like exactly the kind of tool we are looking for at my full time job!