Keeping Project File Small for Feature Documentary

[denisd888]

Hello, I am starting an edit on a feature length documentary on a MacBook Pro and I am worried that in time the project file is going to grow so much it will become impossible to work with it - as I have come across a lot of stories of editors complaining about PP for feature lenght projects. The number of sequences I have is growing by the day and I want to avoid a situation where the project will become impossible to edit - and as we are working with lots of markers, I really don't want to end up in a situation where I have to rebuild the project and lose the markers or something similar. I wonder whether someone who understand the workings of the software has any suggestions on feature length projects in PP?

I know some people actually decide to have separate project files for different parts of the film, but I am at the rough cut stage and that just seems extremely limiting - I want to be able to move things around and have flexibility to explore different ways of telling the story. 

I am using proxies (ProRes Proxy 960x540 .mxf) for everything and at the moment it is running very smoothly, but I will be getting quite a lot of extra footage from the director and worry what that could mean. Also, I should mention that the footage is from a bunch of different cameras (mainly FS7 and A7S) and in different sizes (HD and 4K).

I would really appreciate any advice - I want to avoid having to go to Avid, but a lot of people are suggesting it, so I am on the fence as to what I should do - whether just build it in Avid while I still can, or stick with Premiere - but potentially run into problems later.

MacOS 10.15.2

Premiere Pro 14.0.1

I have about 2TB of footage - but will be getting about 3TB more later on.

Thank you very much, any help will be greatly appreciated!

[Phillip Harvey]

I can't give a lot of advice, but I worked on one feature length documentary style project with Premiere and while yes, load/save times did tend to increase, I didn't encounter anything showstopping. I'm trying to remember if I had it broken into different chunks at any point and it's possible, but it all ended up in one place eventually. This was just me working on it so I just kept very organized and kept a good workflow. It was several TB of mostly c300 footage shot over the course of a year or so.

If you are working with other editors, as many large projects do, then that's typically where people start to prefer AVID. Premiere did just introduce Productions, which I think is supposed to better compete in that regard, but I'm not super familiar with it yet.

[denisad11432051]

Hey, thanks for the reply - for some reason I am not permitted to log in to my main account for adobe community website at the moment but wanted to thank you anyway. Good to know that you managed to do it. Just out of interest, how large was your final project file?

Yes, I am aware of that situation with the project sharing issues - as far as I can tell apparently it doesn't work very well with large projects - at the moment it isn't something that's necessary, but I worry that at some point if other editors get involved, we might have to migrate the whole thing precisely because of this issue.

It seems like PP doesn't really have the feature market worked out very well - I have ended up doing in PP because I just much prefer it to Avid (and the community support is great) and also because people mention Deadpool and a few other examples, so it sounded to me like PP is establishing itself as a viable feature film alternative, but I am not so sure anymore! I guess the editors who did those films had loads of Adobe support and great technical setups which isn't really my case.

Anyway, I will see how it goes but if anyone else is able to chip in with their advice or views it would be very helpful!

[Phillip Harvey]

I'll see if I can dig out a drive today and look at what the Project File ended up being.

[denisad11432051]

Great, thank you very much Phillip!

[Phillip Harvey]

Looks like about 2 TB worth of media. The project file was about 265MB.

[denisad11432051]

Great, thanks for getting back to me. That's a pretty large project file, good to know you didn't run into any problems with crashing etc.

[MyerPj]

One thing that blows out the project size is Warp Stabilizer as all the warp data is stored in the project. (my guess, a bad decision made early on). So, I usually warp the clip in After Effects then export out a ProRes or Cineform intermediate and use that clip in PP.

[denisad11432051]

Yeah, thanks - I will keep Warp Stabiliser in mind - I have noticed it before!

[R Neil Haugen]

Do NOT stay with the Olde Style single-project file process. Period.

Use the new Productions process. And it is WAY different, which is why it works so much better with large projects and masses of assets.

The manual is here ... Productions Reference pdf  

This page talks a bit about it ... https://theblog.adobe.com/introducing-productions-in-premiere-pro/

I've been using it for a while even in my small shop, and it's a pretty amazing process. Took awhile to really accept the changes. I suggest you think of your main project as X Job, rather than as a project. It's less confusing ... because The Job will have a ton of itty bitty project files in a slew of subfolders.

Among other things ... this is built on a folder tree on disc, you create a Production and it asks where to create that Production's top folder ... you name & place it on your machine.

From then on, you add folders inside the Production folder to organize your project, with subfolders as needed. And you create project files in those subfolders for bits of the project like one for sequences, one or more for media ... and if that sounds weird, understand: they suggest in your mental and physical organization of The Job you use folders to give order, and project files to hold assets. In other words, project files essentially become bins for holding assets such as media or sequences.

You copy media from the media project to the sequence within a sequence project. Or drag/drop. It's easy. It does NOT create any media 'assets' in the sequence project, it's all linked internally back to the project that asset "came from".

No dupes ... and far less project ever loaded into RAM/cache files at any one time because The Job is split among many project files and you're only working with one or two bits at a time.

This is the process the Hollywood team has been using for the major motion pics. It's why Premiere had the simple majority of films at Sundance this year ... over 50% were cut in Premiere, with Avid & Resolve fighting it out for a distant second/third. There are of course locking for any project file currently in use so this works very well for users working on a shared storage system. You can look at and copy things from a project file currently being worked/"owned" by another user, but you can't write changes to it until they close that file on their machine. And their user name will show on your machine by the project file/s they have open.

Neil

[denisad11432051]

Great, thanks so much, this is exactly the kind of answer I was looking for! Really helpful - and a pretty big workflow change to anything I am used to! But I didn't realise this is how they edit all the big movies. Also didn't know PP was used for the majority of Sundance films, that's a pretty unexpected stat for me.

I am definitely going to investigate this - I have seen some people complain about the workflow online, but I guess you get that with everything!

Thanks!

[R Neil Haugen]

The changes they've come up with via their "Hollywood" group are on the whole quite useful. And yes, they have a dedicated team that works with the big-budget feature use, you can say search online for Karl Soule and get a ton of videos of him presenting various parts of their Hollywood workflow processes, and introducing those changes over the last maybe three major releases of Premiere.

Productions ... that bent my mind a bit to get used to working in it. Not that it's perfect of course, but I've spent enough time talking with users of all these apps to know ain't none of them Perfect. The whole make subfolders for organizing the project, with project files in them used the same as you would individual bins in a stand-alone process took me awhile to get used to.

It may take a few days to really start getting this ... it's that weirdly different. But once you go down that road, you ain't gonna wanna go back ...

Neil

[denisad11432051]

Thank you again Neil, you have given me some really great info and guidance on the issue, and answered my question! I am going to investigate this option thoroughly and try it all out soon.

Best,

Denis

[R Neil Haugen]

All the best. Organzing over there is so different at fist that it seems weird ... but you can get used to it pretty quick. It does handle large projects very well, and also for a small solo shop like mine, can even say hold your whole year's worth of jobs so you can access all your b-roll, audio files/effects, all your built-up specialized sequences and adjustment layers.

Neil

[andymees@aje]

Hey Denis

I convereted my current short (30 minute) documentary project to a 'Production' recently, following the reference documentation that Neil has already linked above. It was a pretty straightforward process and my efforts were rewarded with a much snappier 'Current Edit' project... but be careful not to guzzle to Kool Aid too quickly. For my preferred workflows, I did run into a few caveats, after the fact. Not showstopping problems by any means, but I'd say that knowing those limitations, now, will certainly help guide how I set up such my 'Productions' in future.

1) Search Bins will only work at a project level, not a production level. If you're embarking on a project that will contain vast numbers of clips, and if in your workflows you use metatdata to log/tag and organise that media (through Search Bin functionality), then be aware that all of your production’s media (or all that you need to be able to search at any one time) will need to live within a single project… so, for example, don’t break the rushes down into ‘dailies’ sub-projects if you want to be able to search across multiple days.

2) Splitting sequences into different projects currently prevents the ‘Reverse Match Frame’ function from working. ie A match framed clip from a sequence in one project cannot be reverse match framed into a sequnce in a different propject. Hopefully that’s something that can be fixed, but if it’s a function you need now, you need to keep all sequences that need to interact in that manner within the same project.

3) Clip usage figures are calculated ‘per project’, not ‘per production’. If you tend to use Video Usage and/or Audio Usage metadata to track and guide media usage in your edits, you’ll need to keep the edit(s) you’re tracking in the same project as the media.

4) Nested Sequences need to live in the same project as the Master Sequence(s).

There may well be other advantages/disadvantages... everyone's workflows are different, so the process will obviously affect different users in different ways... but those are the ones that jumped out for me.

Cheers

Andy

[denisad11432051]

Hey Andy,

These comments are actually extremely useful, because it's exactly this kind of stuff that can really complicate your life. I wonder how well markers work in this workflow - I presume probably not that well. I think we will be setting up reels for every single day, so it's good to be aware of that limitation. I guess one of the issues with this is that it's still a relatively new way of working so perhaps not everything is seamless yet. I am going to investigate it and make a decision in the near future, but thanks for bringing up these issues!

Denis

[R Neil Haugen]

GREAT post about the limitations of the Production process as-is.

The metadata tracking issue is one I'm struggling with. I know Karl Soule has emphasized in his NAB and other live presentations that the Hollywood crowd needs hefty meta-tracking. But as you noted, the way we're supposed to break projects up for best playback performance busts up meta-tracking.

Um ... we need both, don't we?

Neil

[MyerPj]

Very interesting thread!