Long Form Project Optimisation

Hello @Theo30896725bss4,

Thanks for your message. It's been a while since you filed a bug report. Is the performance issue you originally filed solved? If not, can you let us know?

Barring the other things mentioned here, I recommend turning off captions to troubleshoot long-form sequences' performance issues. Let us know if that helps.

If you have a new issue, feel free to file it here. See: How do I write a bug report?

I'll move your report to the Discussions forum while we await your response. I'm sorry for your frustration on this issue.

Thanks,
Kevin

Another update in this odyssey here: So now it seems that there is actually some kind of problem with the actual computer the lead editor was editing on, which has been causing all the performance issues, as we switched him to the other workstation and are now encountering no problems with the master sequence (few hours in now). We have three 2022 Mac Studios in the office, Apple M1 Max 64 GB Ram, running Mac OS Monterey 12.4. Is there any possible known issue with compatibilty that could be causing the issues between the Mac Studio and Premiere? Tomorrow I will have to compare both systems to look for any possible differences that could be causing the problem but I'm a bit stumped, and not much of an IT guy if I'm being honest. 

>The database thing seems to make instant auto-save easier and thats how team projects behaves.

Yes, and...auto-save is now also basically instantaneous in PPro for .prproj projects as well, because of work we did in 23.x. 

You're right, moving to a database model (which we're working on) would help accelerate incremental changes. But it won't do anything to prevent the necessity of loading (for example) Warp Stabilizer's <ahem> "hefty" amounts of sequence data. 

>...what is going on to slow down timeline interaction on big projects!

Specific causes vary; that's another of those truisms / laws of physics ("complexity is complex"), that we're unlikely to bypass any time soon.

We are happy to pursue specific, readily-reproducible cases ("This sequence in [this project], relying upon [this media], performs really badly") further.

This is a great discussion honestly, pretty enlightening. 

I don't even bother with multicam sync in Premiere, but I agree re: merge clips. A real sub clip workflow (not to beat the dead horse, but as in Avid) would be helpful. 

We sync dailies with tentacle sync, then import the xml and do all the more manual customization inside of Premiere for daymaster sequences. I know multicam sync but just don't really enjoy using it all that much. 

Re: my subtitle woes, I cooked up a workaround that seems like it could make difference performance wise, but I will know for sure once the lead editor sits down to work tomorrow. 

I printed the subtitles for all the spanish scenes as alpha channel, then threw it on top of the master sequence as an overlay and cut out all the spots that don't require subtitles, then deleted the subtitle caption track entirely. 

It's my experience that we encounter performance issues even when the subtitles track is disabled, or all the subtitle captions are disabled, so I'm interested to see if this makes a difference. Obviously the captions are then "burnt in" as an overlay, but we are far enough along in the edit at this point that changes to the subtitles themselves will likely not be necessary. 

It's not super ideal obviously, but I'm very curious to see if it fixes our issues, as I am more or less 85% sure that the large amounts of subtitles are the biggest performance bottleneck in our master sequence. 

Yeah I'm not a programmer. The database thing seems to make instant auto-save easier and thats how team projects behaves. In a normal PrProj file the whole things seems to be need to written out, which is a killer on jobs with a lot of Warp Stabilization. 

In my mind the database would allow for smaller incremental changes to the object in memory, say if you need to move 1000 multicamera clips down the timeline 10 seconds to open up space for a new clip in an 90 min long timeline. Instead of parsing 100s of megabytes of information about media location and warp stabilizer data etc to write out the new project file (in memory or disk)  it would just update the timeline information.

I'm sure this is some version of this is how it DOES work but to the user it seems like project bloat kills performance, so it begs the question what is going on to slow down timeline interaction on big projects! Why is moving clips so hard! Even when you spend a lot of effort moving all your media to sub projects within a production etc, once a timeline reaches a certain complexity all hell breaks loose.

> Either fix merged clips so they can work with turnover workflows or give us a solution that works!

Merged Clips is on our list for an overhaul; not sure if/when we'll be able to obviate the laws of physics, around multicam sequences being demanding. 

> My general understanding of some of these problems in PrPro is that at the end of the day is that it stores everything in a giant XML file in memeory and so once the project gets to s certain size things start to lag and bog down really badly.

That's close to correct; PPro does use XML for project storage, but once the info stored in the (xml-based) .prproj is in memory, that memory representation is exactly what would be read into memory, from a Team Project (or any other database). 

>...But it doesn't quite explain why I would see differences between my media living on my RAID storage vs. my internal SSD

Correct, project storage format (XML or database) has nothing to do with media access speed. 

I would guess it has to do with total IOPS. SSDs are the fastest when it comes to just pure IO. RAIDs have to go through a bit more overhead I would suspect to read a file based on how parity information is stored and calcuated? Probably depends on the quality of the RAID controller used in the device. Also that info is going to need to pass through a few more paths to get to the CPU where I think in the new Macs the internal storage is nearly as fast as RAM for the CPU to read. 

I don't have sources this is all hearsay and speculation 🙂 . 

> I would love to learn where I could find more valuable resources about the inner workings of PP/editor optimization

This is probably the best source Adobe provides:
 
https://helpx.adobe.com/premiere-pro/using/long-form-episodic-best-practices.html

@Scott.C. If your understanding about reading a giant xml file is correct, it explains A LOT.  But it doesn't quite explain why I would see differences between my media living on my RAID storage vs. my internal SSD, when both are more than fast enough to serve all the video media on demand (although obviously the SSD is faster).

My general understanding of some of these problems in PrPro is that at the end of the day is that it stores everything in a giant XML file in memeory and so once the project gets to s certain size things start to lag and bog down really badly. I think other NLEs have moved to SQL databases etc or have ways of mitigating this need to read and write a 1GB (probably the size once it is uncompressed into RAM) project file for every little timeline edit. I know teams projects uses a database format I always sort of hoped they could optimize that and just make it the standard project file.