RAM maxing out after relinking

[Pyromk1]

Upon opening a couple of my files I am watching my RAM usage hit the 90's whilst I watch the re-linking circle fill. Ususally this drops back down to acceptable levels. With a number of my projects the RAM usage jumps back up to 99% after a few seconds and stays there for a few minutes until I get "Premiere is running low on resources" messages. If i leave it long enough I get a black screen crash.

I am not attempting to do anything whilst this happens. There are no timelines with anything on them.

Typically these projects consist of wedding footage, up to about 800 clips of varying length shot in 4K h.264

The only note i can make is that The first time it happened it was a file originally created on my laptop. The last time it happened I accidentally opened the project before powering on the DAS. I immediately realised the issue when the relinking screen popped open and closed without saving. This should not make a difference but I'm really grasping at straws for a solution here.

The only workaround I've found has been to delete the project and start again. luckily the projects that have had this issue recently have been right at the start of the project so it's been an annoyance but not catastrophic. I'm worried thought that it's only a matter of time before this happens to a project that I'm half-way through...

My setup is:
Premiere Pro 24.5
Programme on C drive, Project files on disk A, cache on disk B, footage on a 16TB DAS (with 1TB free) in Raid5 (QNAP TR004) served by a USB 3.2 2x1

RAM: quad channel Corsair vengence LPX 64GB DDR4 3600mhz
(Processor: i9 10900K, board: Asus Z490-A, graphics: RTX 3080)

[R Neil Haugen]

The fun part is that your media is resource-intensive to work with. Eight hundred long-GOP clips is a load ... as to simply show the next frame, the computer has to decompress up to 100 or more frames, and store them in RAM/cache files.

Now try and play that on a timeline, yea, it's a load. Your system has a decent 10-core CPU with QuickSync, so it's better at this than many. But still ... that is a ton of long-GOP media to work with.

I work for/with/teach pro colorists, most work on 'heavy iron' that makes my 24 core 128GB of RAM system look positively anemic.

And none  of the many colorists I know would ever grade a program of several hundred long-GOP clips without at least proxies ... most would simply t-code to ProRes before conforming the project into Resolve or Baselight. That's how poorly they view working with long-GOP files.

Even a few in a project, they will normally t-code to a workable intraframe codec.

We all wish it were otherwise, as long-GOP is so fast for the camera maker's to get encoding to small file sizes in camera. My Ursa Mini Pro 4.6k, even doing it's most compressed BRAW, still fills cards quite rapidly. Incredible media quality, but file size is not ... small.

It's one of those many trade-offs of Life. Maybe @RjL190365 could have some advice, as they know this hardware stuff so major. There might be something in your setup that could be tweaked.

[Kevin-Monahan]

Hello @Pyromk1,

I'm Kevin from support, one of the moderators here. Thanks for the bug report. The information you provided is very helpful to the team.

Neil's advice is very good. You might also try a test where you work with a sampling of the media locally rather than over a network. Let us know if transcoding or working with the media locally might help bring about a solution.

I hope the team will respond to this report shortly. Sorry for the frustration.

Thank You,
Kevin

[Pyromk1]

Thank-you @R Neil Haugen  for such an in-depth reply.
I am reminded of how much of B h.264 is for decoding every time I use thumbnail view! We uses mainly Sony Alpha cameras for our work which have very few codec options. I also made the decicion that in terms of workflow I'd same time and storage building a machine that is capable of working with long-GOP clips rather than transcoding.
It has perfomred admirably until now and I'm struggling to understand why.

@Kevin-Monahan Thanks for your feedback. I'm actually using a Direct Attached Storage with a 10gb connection rather than Network Attached Storage. So the footage is already "local".
I've always transcoded to proxies when ive needed to from within Premiere. But as I can't open the project in premiere that wont be possible. Can you point me towards any articles detailing a workflow for this from (i'm assuming) media encoder. Then how do I get permiere to perform the final renders from my originals ?

In the meantime I've got a 128GB RAM kit on it's way to see if that is enough RAM for premiere.  

[Kevin-Monahan]

What happens if the media is living on internal drives, @Pyromk1? Better behavior or no change?

Thanks,
Kevin

[Pyromk1 · Accepted solution]

The thought had crossed my mind. The current project files that I'm using no longer behave in the same manner. I deleted the problem projects and started from scratch. I did find that attempting to import all of my colleague's footage in one go (470 files) created the same problem in one of my projects—but only his footage. Importing in batches of 40-50 was no problem.

Once the problem rears its head again, I'll try shifting footage to an internal drive to see if that makes a difference. Then, I'll let you know my findings.

[Kevin-Monahan]

Thank you so much, @Pyromk1! Moving to the "Discussions" forum while we await any further troubleshooting.

Kevin