Copy link to clipboard
Copied
Hello,
I work as a Post Supervisor and Assistant Editor and I am looking for a real guide to optimizing Premiere Pro for long form (1.5-2hours). I have read the Long Form and Episodic Workflow Guide developed by Adobe cover to cover (which was very useful), but we are still encountering significant software performance issues when our master sequence for the films become larger.
The editors are working with the latest version of PP, on Apple M1 mac studios, 64 GB ram connected to an Editshare server with 10GBps connection.
I have the media cache on a Samsung T7 Shield 1TB SSD.
Then all the media is seperated out according to Adobes specified production workflows (seperate projects for each shooting day of material, etc.)
The editors are cutting with Pro Res 422 Proxies.
It still becomes suboptimal when we start to encounter playback lag, slow actions, constant beachballing for 2-3 seconds that quickly becomes extremely inefficient and frustrating for us to edit with.
We do not work with burnt in LUTs, rather the LUTs are added to the proxies with Lumetri Color to give the editors more control, maybe that could negatively impact performance?
There are also large amounts of subtitles inside (the project is a documentary featuring some sequences in Spanish) could that also be an issue?
In any case if anyone could enlighten me on something I am missing it would be an extreme help, because I feel like I have tried everything at this point and I am starting to think editing larger productions with Premiere is much more trouble than its worth.
Thank you!
Hi Theo,
Do you have a project & media which, when you follow [certain steps], reliably displays the sluggish performance you've described? If so, we'd like to get those materials, and (crucial!) those steps to follow. Private upload space available, upon request.
Regarding editing larger productions: From what you've written, I don't think project size is relevant.
Of course, for troubleshooting, actuals always beat speculation. 🙂
Ryan, your concerns about cache efficiency are valid, though pote
Copy link to clipboard
Copied
What resolution are your sequences and original source media? Are your editing sequences, using proxies, working at the same resolution as the original?
Your description gets at a general frustration I experience often, whenever sequences start to get long, especially if multicam clips are involved. It seems that whether my media is on SSD vs. HDD RAIDs makes some difference, not b/c the RAIDs are struggling in the slightest to support the data rate, though (everything is plenty fast). I know this sounds absurd, but it almost feels like Premiere wants to access everything in a given timeline at once, rather than just the upcoming frames that it needs for playback, and the longer the sequence, the more it chokes on doing anything, even if everything is proxy'd or rendered, or whatever.
I worked at a facility some years back, in my earlier days working with premiere (2015, maybe?) with Mac Pros on a Fiber SAN and remember being told to make sure my media cache was on my internal drive (which I know is good common sense but that's not my point) and I asked why, and what i was told was that the SAN was plenty fast enough to play back our footage (it was XDCam 720, I think) but that what the SAN struggled to support was the number of requests that Premiere makes at any given moment for media cache files. The cache files are obviously tiny, but still required requests to be accessed, and so the SAN would choke not because of insufficient bandwidth to move large files, but b/c the server couldn't handle the thousands of requests for tiny files. Now I may be misunderstanding exactly what I'm talking about or mis-extrapolating, but I think about that a lot: that is, the difference between the bandwidth needed to playback a large video file, and the traffic jam of requests to just check on files that may actually require virtually no bandwidth but do require traffic. It seems to me that there might be some deeply embedded inefficiency in how premiere handles sequences or projects with a lot of footage. Like, it's checking in on all the files in a project or sequence more often than it seems like it really should need to? I don't quite know what I'm talking about but I do know that it makes no sense when I'm on a fast RAID, on a fast Mac with tons of RAM, line-cutting a basica mutli-cam sequence, and things just start to get to stupid slow once the sequence starts getting much past 10 or 15 mins in length.
Copy link to clipboard
Copied
Hey Ryan,
Thanks for the quick reply. We are editing in a HD sequence with HD Pro Res 422 Proxies, that I transcode myself in Davinci Resolve from the original camera material. I just noticed our preview files for the master sequence are set to Pro Res LT instead of Pro Res Proxy, would that make any difference? I doubt it but just curious. The original source camera files are in UHD, but we edit in HD as we will be delivering in HD. With our workflow the original camera clips do not come back into play until turnover to Color anyway.
We have little to zero nested multicam, However many clips are stacked on top of one another currently as we are still deep in the edit and a few weeks away from picture lock.
I ran a disk speed test on our Editshare server, it hits around 520 MB/s write and 1020 MB/s read. That should be sufficient.
Everything I have read from Adobe states a SSD for the media cache (seperate from boot drive/premiere itself) is the most efficient setup.
It is frustrating to say the least, as I believe I have followed Adobes rules to the letter with this project, and we are still encountering similar performance issues. I am getting close to abandoning Premiere as an option for long form projects all together and just sticking with Avid, but I know David Finchers team has been editing their features exclusively with Premiere since Gone Girl, so there must be something I am missing, its just impossible for me to really say what it is.
Copy link to clipboard
Copied
Hi Theo,
Do you have a project & media which, when you follow [certain steps], reliably displays the sluggish performance you've described? If so, we'd like to get those materials, and (crucial!) those steps to follow. Private upload space available, upon request.
Regarding editing larger productions: From what you've written, I don't think project size is relevant.
Of course, for troubleshooting, actuals always beat speculation. 🙂
Ryan, your concerns about cache efficiency are valid, though potentially misapplied.
Useful metaphor: While storage vendors, like car manufacturers, like to brag about maximum top speed and 0-60mph times, drivers spend a lot more time stopping and starting around town (rarely making it to 60mph), before they make it to the (final output) 'freeway', where they can reach those quoted top speeds.
It's the same with PPro's media access. Yes, the conversation between PPro and all active media files in a given sequence can be 'chatty', and very often pertains to topics other than "Give me video frames, in linear time sequence, as quickly as possible." Those sub-optimal conversations remain necessary.
Also (hopefully) encouraging: A member of David Fincher's editing team recently joined the PPro Engagements team. 🙂
Copy link to clipboard
Copied
Yeah, I don't know what to say other than to kind of join the complaint. It just shouldn't be so hard for Premiere to work with large projects that have lots of footage, especially when using productions and proxies.
I tend to favor using 960x540 for proxies, which is a little pixelated if I start doing punch-ins on 4k footage (which I do in a lot of 1080 deliverables for corporate clients).
I did cut a feature doc last summer on my Mac Studio M1 Ultra, 128 GB of RAM. I used a Production, worked in UHD the whole way thru, with 540p proxies attached to all the original footage, in multicams, and I was pretty pleased overall with the performance (compared to my 2013 Mac Pro which I only got rid of last year). Footage was on a 4-bay Pegasus RAID, with proxies on another 8-bay RAID. For the most part, I could scrub the timeline like it was SD. But then I'll work on smaller projecst from time to time, like a 3 camera line-cut of a 1 hour panel discussion I was doing recently, and things started getting stupid sluggish about 10 or 15 mins into the sequence. I don't understand what's going on. There's just something weird about why a multicam sequence that plays like butter before I start editing, suddenly gets sluggish and draggy and beach-ball-spinning once I've made a few dozen edits in a timeline with it. But I've observed it happen many times on many different sorts of projects with differnet sorts of media. For this particular panel project, I found everything improved a lot when I moved my media and proxies off my 8-bay RAID onto the internal SSD, which is weird b/c my RAID is plenty fast enough to support the 3 streams. I just don't understand why premiere sometimes seems to want to do a ton of extra work to play things that don't seem complicated. My solution is usually to just segment projects a ton in productions.
One pattern I've noticed is that the amount of footage in a project (or in a Production, the total amount of clips of footage that are open across multiple projects) seems to have an impact, even if my sequence(s) aren't complicated. Like I've had projects where if I have a project in the production with, hundreds of clips in it, whenever it's open, everything else runs slower. But as soon as I close it, everything runs better. So practially-speaking, it seems to help to keep those individual projects within the production as small as possible, and open and close them as I need them.
One other recent experience I've had: my wife is cutting a feature doc right now on a MacBook Pro M2 Max, in 1080p, with 540p proxies off of 1080p source footage, and was having massively sluggish behavior with some long select sequences. We shortned sequences and set up a production, etc, and it helped but was still slow. But once we moved her media drive (a single USB HDD, but still perfectly capable of the throughput she needed) off of an extneral OWC Thunderbolt 3 dock and directly onto one of the TB ports on the Mac, everything transformed, and Premiere runs great now (still using a Production, of course). Obviously there's something funky about the TB dock there, but that's why I think about the requests to the SAN story that I mentioned earlier; her drive had just fine throughput on the dock when we tested, but perfomance was totally different once the dock was out of the picture, and I think about the # of requests or something. Maybe I'm combining 2 things that are unrelated, but it feels potentially similar. Regardless, I think we still had her cache on the internal SSD.
As far as Fincher, I don't know much but I know his team does a lot of customization. I went to an LAFCPUG event with his post team back when he did the Zodiac film (still somewhat early days of all-digital workflows in motion pictures - maybe in 2007 or 2008?), and I think they might have been using FCP 7 at that time, but point is, they had a dedicated software engineer on the team (or at least, he seemed like one to me) who spent all day writing code or scripting to generate proxies for editing and move around media. So like.... they were operating with more robust technical resources than the average production, and following some very customized workflows.
Copy link to clipboard
Copied
>It just shouldn't be so hard for Premiere to work with large projects that have lots of footage, especially when using productions and proxies.
For most users, it isn't; happy to help further, provided some actionable specifics.
>There's just something weird about why a multicam sequence that plays like butter before I start editing, suddenly gets sluggish and draggy and beach-ball-spinning once I've made a few dozen edits in a timeline with it
We fixed a problem between 24.1 and 24.2, which could have been the problem you've described. Also, multicam sequences are a great way to make very demanding sequences.
Also...which "TB dock" did you use (and encounter trouble with), on your wife's production? My CalDigit T4 seems issue-free...
While the Fincher team does rely on custom panels to enforce workflows (and keep metadata up-to-date), I can confirm that they're using the same PPro as everyone else; no custom versions.
Custom panels have zero impact on timeline performance, or data throughput.
Copy link to clipboard
Copied
Thx Bruce; I hear you, and I like your metaphor. I guess my only point was that it's easy for us as editors to say "oh my storage is fast enough to play back 6 streams of this or that hi-res footage in RT" but that's not the whole story; those litlte start-n-stops as you describe are obviously vital. And maybe what I'm wondering is whether a storage system that's fine at playing back X # of streams of high resolution may or may not be good as servicing the all the start-n-stop traffic. It's out of my depth, but thinking about stuff like the size or number of packets that are being passed thru the 10Gbe?
So to apply those ruminations practically here... @Theo30896725bss4, have you been able to test any particularly offending projects or sequences in a situation where the EditShare is out of the picture? Do you experience sluggishness at the same point if you test working with the footage off of an internal or local SSD as opposed to the EditShare? I wouldn't think that SHOULD be a problem, but maybe in fact, it is?
Copy link to clipboard
Copied
Hey Bruce, my wife is/was using an OWC 14-port Thunderbol 3 Dock. I think part of the issue there may be that she had a monitor running off of it too. The mnonitor is working fine and she has some other drives on it currently, but we weren't doing anything that's unsupported, as far as I know.
TBH, this sort of thing with big projects getting crazy sluggish goes back years for me in terms of versions. I just kind of accept it, and like I said, love using Productions as a way to streamline just how much media I have open at any given time, which seems to have greatly reduced how often I encounter it.
For example, I had a large project about 6 months ago, where I was using a production, but I had one partuclarly large bin of broll footage. I'm going to guess it was around 500 clips. And whenever I had that particular project open, everything in my editing sequences would slow down, even if I wasn't accessing any of the footage in that particular project. If I closed that project, then my edit sequences (which were in a separate project in the production, and mostly under 10 mins in length), would behave mostly fine again. Is that expected behavior?
The panel discussion project I was mentioning earlier: I will test re-linking it to the media on my RAID rather than my internal SSD and see if that sluggish behavior resurfaces. I don't recall what version of Premiere I was on when I started working on it (I know I'm on 24.3 now). I will circle back if I can reproduce.
Copy link to clipboard
Copied
Soudns good; thanks for the additional details!
Copy link to clipboard
Copied
Hey guys,
@Ryan Fritzsche to answer your question, I have not yet tested working the production exclusively out of an SSD, I could try it to see if the Editshare is causing the issue, but that would be quite shocking to me, considering this is what the product is built for (and was also a solid 15k euro investment).
@Bruce Bullis here is our production panel is it readable?
I have organized all the source media from each shooting day into seperate projects for each shooting day, in addition, the source clips are seperate from the Daymaster stringouts of the footage that the editors have been pulling clips from to create the master sequence of the film. With this workflow, everything was running quite smoothly until we hit the 1 hour, 1.5 hour mark (the final film will be around 1.5 hours) in building the master sequence. Then the lead editor started running into problems when cutting the master sequence.
Also to note, the proxy clips themselves all have a Lumetri Color effect on the source clips for the LUT we are using.
In general I try to keep almost every video clip used in the edit as a Pro Res Quicktime, as h264 is not ideal.
As it stands currently the editors are able to work, but we are losing significant amounts of time due to small slowdowns (1-5 second delays for small actions, etc.), and its frustrating to work with overall.
Regarding sending materials, Im not sure how I would be able to upload everything, it is quite a large project (almost 1 TB of proxies alone).
I have ran into issues before with Premieres caption workflow when it comes to slowing down a sequence, as we have quite a large amount of subtitles inside the project (half of the spoken dialogue is in Spanish). Could this be a cause for the slowdown?
Thanks for the input!
Theo
Copy link to clipboard
Copied
Hey Theo, I stand by my own frustrations with this kind of thing broadly-speaking but I also have to say, if you're only starting to run into any slow downs at the 1 hour mark, that doesn't seem all that bad to me. On complex projects, I have in the past gone old-school and worked in "reels" (ie: break the whole film into 20 or 30 min chunks, in separate timelines/projects), and then just put them all together to export, where a few seconds of lag to copy-paste big chunks is a small price. .
I was working in an 85 min timeline on the doc I cut last summer, and as I said, it was generally pretty much sluggishness-free, but I do think that part of that was becuase the film had shot relatively little footage (as feature docs go), and certainly torwards the end of the process, I did have some beachballing and slowdowns here and there, which didn't surprise me too much. What was aggravting was how the slowdowns seemed kind of random, in that I couldn't easily pinpoint what circumstance had changed when things suddenly got sluggish. Sometimes just restarting Premiere would suddenly breathe new life into the performance, which was weird, and other times I had no issue scrubbing around the timeline.
Copy link to clipboard
Copied
I can see the screenshot, thanks.
What version of PPro are you running? [from the About box]
We'd be curious to learn whether disabling all those Lumetri instances applied to the proxies, perceptibly improves performance...?
Also: Yes, subtitles can definitely slow down performance.
Copy link to clipboard
Copied
Got it. We are running Premiere 24.1.0. Re: subtitles, I figured, but what other workflow is there for editing a project featuring a language the director/editor can't speak? In Avid it is never a performance issue to work with subtitles. We tried disabling all FX globally (Lumetri included) and saw no significant performance improvement.
@Ryan Fritzsche the reels would definitely help, I will run that by the editors tomorrow if its an actual suitable soluiton. Think the issue is that becomes cumbersome for the director and other editors to get a better feel for the film as a whole as in being able to scrub through and watch everything at once.
Is the media cache with regard to SSD or boot drive a bottleneck? I know a SSD is recommend, but I cant find any real information exactly why.
Copy link to clipboard
Copied
Quick update if interested: to our astonishment today: we quit premiere, unplugged the audio interface connected to the edit workstation of the main editor, plugged it back in, restarted the production and then deleted the entire media cache, forcing it to rebuild. The edit of our master sequence is now running quite smoothly since. I have no idea why or how the audio interface was causing an issue, hardware compatability issues are much more of a historically Avid problem than a Premiere one, but in any case so far everything is running much faster and smoother than before (big knock on wood).
Copy link to clipboard
Copied
That's amazing. Glad to hear, but so weird.
Copy link to clipboard
Copied
Seems like I spoke too soon. It was running well for most of the day, then we started to run into similar small moments of lag/delays for the smallest of actions towards the end of our work day, tried unplugging the interface and plugging it back in again but no difference, bummer.
Copy link to clipboard
Copied
Hey @Bruce Bullis I'm not sure if you're the right person to ask, but I would love to learn where I could find more valuable resources about the inner workings of PP/editor optimization, I feel like I've exhausted most of the videos I can find on youtube, etc. and I've read the PDFs, and most of the videos I find now when I'm looking for something more specific have little to do with the optimization questions I have, as in, they are often more directed towards beginner level users/much smaller projects than what I am typically working with. If you happen to know of anyone/anything let me know!
Copy link to clipboard
Copied
Also not sure if you guys are still around/can answer this, but here is an example of a cut of the master timeline, is there a reason why the preview files do not seem to render? as in the entire sequence is yellow. The preview file format is set to Pro Res LT, could this impact performance?
Copy link to clipboard
Copied
TBH, I feel like Premiere Pro has never really excised its render file demons. It used to lose render/preview files for no apparent reason, and while that seems to happen way, way less in more recent versions (hurray!), it still happens with clock-work regularity when I work with subtitles. Most common example is that I'll render a sequence with subtitles, then go to export, and almost immediately, half of the subtitles will become unrendered. So my hunch here is that if you are using subtitles on a substantial portion of your footage, you will lose your render/preview files very shortly after rendering, anywhere the subtitles are involved. It isn't consistent in my experience - sometimes rendered subtitles stay rendered, but most of the time, if I render them and then make any change anywhere in the timeline or close/reopen the timeline or project, they will un-render. It's often felt to me like something about how Premiere deals with fonts, but I'm not sure. I do know subtitles virutally never stay rendered in my experience.
Copy link to clipboard
Copied
Ok thanks @Ryan Fritzsche good to know.
Just did a test and seeing exactly the issue you are talking about. Overall I find working with subtitles in Premiere to be incredibly painful. I think for international projects this workflow doesnt make sense, and as I far as I know there is no 3rd party solution for working with language translated transcriptions, subtitle editing etc. With Avid it is never a performance issue to edit with subtitles.
Copy link to clipboard
Copied
"We fixed a problem between 24.1 and 24.2, which could have been the problem you've described. Also, multicam sequences are a great way to make very demanding sequences. "
I find this frustrating as this is the only way to reliably sync dual system audio / video in Premiere, which is a must for documentary or really any production worth its salt!
Either fix merged clips so they can work with turnover workflows or give us a solution that works!
Copy link to clipboard
Copied
My general understanding of some of these problems in PrPro is that at the end of the day is that it stores everything in a giant XML file in memeory and so once the project gets to s certain size things start to lag and bog down really badly. I think other NLEs have moved to SQL databases etc or have ways of mitigating this need to read and write a 1GB (probably the size once it is uncompressed into RAM) project file for every little timeline edit. I know teams projects uses a database format I always sort of hoped they could optimize that and just make it the standard project file.
Copy link to clipboard
Copied
@scrozier If your understanding about reading a giant xml file is correct, it explains A LOT. But it doesn't quite explain why I would see differences between my media living on my RAID storage vs. my internal SSD, when both are more than fast enough to serve all the video media on demand (although obviously the SSD is faster).
Copy link to clipboard
Copied
> I would love to learn where I could find more valuable resources about the inner workings of PP/editor optimization
This is probably the best source Adobe provides:
https://helpx.adobe.com/premiere-pro/using/long-form-episodic-best-practices.html
Copy link to clipboard
Copied
I would guess it has to do with total IOPS. SSDs are the fastest when it comes to just pure IO. RAIDs have to go through a bit more overhead I would suspect to read a file based on how parity information is stored and calcuated? Probably depends on the quality of the RAID controller used in the device. Also that info is going to need to pass through a few more paths to get to the CPU where I think in the new Macs the internal storage is nearly as fast as RAM for the CPU to read.
I don't have sources this is all hearsay and speculation 🙂 .