100% disk access and 100% RAM usage for no reason, completely blocking my ability to work

[aaronfross]

Show-stopping performance issue here! I am completely blocked from being able to work at all. Premiere is currently using 100% disk access and 100%+ memory usage, sitting idle, doing nothing!

In Premiere Pro 2024, I opened a project originally created in v2023. Was prompted to convert the file format. After doing so, disk access on the drive where the video files are stored immediately skyrocketed to 100%. It stayed that way for a very long time. Two hours later, the drive was still being hammered to death. Completely unable to use Premiere at all, it was dead in the water because it couldn't play any files. CPU usage was near zero.

Resource Monitor showed Premiere reading the source footage files. If it was trying to build peaks and caches, it did so with extreme inefficiency.

Even worse, after closing the project, disk access persisted at 100%. Resource Monitor showed the same few source footage files being read. Eventually Premiere got tired I guess, or it felt it had punished the drive enough, and eventually petered out after about five minutes.

Other projects opened normally, there were a few seconds of heavy disk access but then it went back to normal -- zero access when Premiere is idle.

Of course I restarted the Windows 10 PC, but that made no difference.

Updated to 2024.5, re-converted the original v2023 project file, but that made no difference.

Deleted all Media Cache files through Preferences, but that made no difference.

I did notice that projects using footage encoded with Blackmagic MJPEG codec hit the drive a lot harder than projects using footage encoded with NVIDIA NVENC H264. I guess H264 files don't need caches at all?

The problematic project has about 100 source files, TRT 14 hours, all encoded with Blackmagic MJPEG. Totaling about 400GB, which is kind of a lot, I know. It's showing them all as offline while it hammers the hard drive, presumably trying to build caches and peaks. So one might conclude that there's a performance issue with the codec? It's not hardware accelerated. But there was no issue whatsoever when I originally edited this project in Premiere 2023. And again, CPU usage is near zero, so the problem seems to be completely about hard drive access.

Contrast this with another project using the same codec, with only ten files totaling 3GB, TRT 30 minutes. Even after deleting all cache files, the smaller project takes ~15 seconds to load. The larger project does not finish loading even after two hours. The math here doesn't seem to add up. I could imagine that 400GB would take a half hour at most. Not multiple hours.

I did update the Blackmagic Desktop Video software, which includes the MJPEG codec. Not really relevant, though, because Premiere has native support for MJPEG. It just won't load the audio in the AVI files created by Blackmagic unless the Blackmagic codec is installed. Not a concern for me here because I'm not even using that audio. But updating the Blackmagic software made no difference.

Any way you slice it, a project should not take hours to load, even if the source footage is 400GB. I am really not accustomed to this sort of behavior. Sony Vegas never made me wait for hours while it ground my hard drive to dust, no matter how large the source footage folder was. If this is supposed to be speeding up my workflow, I regret to have to say that it is the opposite of that. It has completely shut me down.

I really don't want to erase my preferences, since it was a huge PITA to get Premiere set up to be even remotely close to an efficient workflow.

So what am I supposed to do here? Reinstall v2023? I doubt that would make any difference, as it seems the issue is just massive inefficiency in building the cache files.

Is there any option to NOT build these cache files? Or to control how many files are read simultaneously? Maybe if it was only reading one file at a time it wouldn't be fighting itself for limited bandwidth?

BTW, the hard drive is not super fast, but it is a 7200 RPM Western Digital platter, internal. Again, if I were to do a backup, I would expect that 400GB would take maybe a half hour to transfer. Not multiple hours!

And why are these caches temporary external files in the first place? If they take so long to build, shouldn't they be persistent? If not stored in the project file, at least stored next to the source footage? That's an option for the peak files. Why not the cache files? Why set things up for failure like this?

In the end, I let the computer sit for hours until the disk access went back to zero. All of the footage in the bins finally read as being found, not pending, and all of the statistics were visible. Tried to access the timeline and BOOM, disk access went back to 100%. What the actual frak?!?! And now RAM usage is pegged at the maximum, too! I have 32GB RAM and have 6GB reserved for other applications. Premiere has gone totally rogue here, consumed 100% of disk time and 100% of memory. Now it's using the page file, completely runaway process, it's currently up to 44 GB and still rising. SITTING IDLE, DOING NOTHING.

I turned off the timeline thumbnails with the super secret hotkey. No change. Disk access still pegged at 100%. RAM usage still greater than the maximum I set aside for Premiere. What is Premiere even doing? It's not building caches, it's not playing the timeline, it's just sitting idle. This is OUTRAGEOUS! Resource Monitor says it's still accessing those source files. But WHY???

How do I get back to being able to actually work?

This is just shocking!!! It's acting like malware!

[aaronfross]

Changed the name of the media's enclosing folder. Opened the project. Converted to 2024.5. Left all files offline. Relinked a single file through the Project window. Disk usage again skyrocketed to 100%, presumably while building cache, but Progress window remained blank.

After waiting at least five minutes for the cache of a 30 minute file to be built, tried to access the timeline. Again was blocked by 100% disk access. After a few more minutes, it finally stopped. Performance appeared normal until I moved the play head to a different point on the timeline. As soon as it touched a second UNLINKED clip, disk access slammed back up to 100%. Waited another five minutes or so, then disk access went back to zero and things seemed to work normally.

Relinked a second file. Waited five minutes for the cache of an eight minute file to build. Tried to access the timeline. That clip played OK. Tried to access the first clip on the timeline. Again blocked by 100% disk access. Premiere was having problems accessing the file that it had no problems with a few minutes earlier.

After waiting another five minutes for the first file to load in the timeline, moved the play head to the second file in the timeline. Disk access again skyrocketed back up to 100%, but only for maybe 30 seconds. The second clip now played OK.

Moved the play head back to the first file. Disk access skyrocketed back up to 100%. Waited another five minutes or so.

During all of this, RAM usage massively increased in sync with the disk access. When disk usage went back to zero, saved the project and exited Premiere to reclaim all of that RAM.

Relaunched Premiere. Disk access and RAM consumption immediately went back up to maximum.

None of this is supposed to happen. My conclusion is that Premiere is broken. Truly, seriously, massively broken.

A shockingly unacceptable situation. Total waste of time and effort. I am truly horrified by this, and it only makes me even more super cynical about Premiere than I was before.

[aaronfross]

Can you explain why a minimum processor state of 100% would cause any problem, or have any relevance whatsoever to this issue?

Can you explain why partitions "make things slower"?

[R Neil Haugen]

As to partitions, they made sense with the older, and far slower spinning disc drives. Partitions did speed data read/write. Which was the only reason for using them.

On the far faster spinners of the last decade, they don't increase access. And of course don't increase access speeds on SSDs. There simply isn't a reason for them anymore.

As to the other questions, we're other users, not devs. And we are not having such issues on anything we work with. So we are trying to do remote troubleshooting to help you.

This seems to be between the specific media you are working with and the project files, perhaps.

[DavidVClarke]

Before you relink to any clips can you look through the project and click on the timeline with no problems?  To me that means Premiere is having a problem loading the clips - which could be a disk/computer problem or could be this version does not like those AVIs.  My next step would be convert one of those clips to a MOV and link to the MOV not the AVI.  Does it carry on working?

[MyerPj]

Wow!

<<Can you explain why a minimum processor state of 100% would cause any problem, or have any relevance whatsoever to this issue?>>

Uh... let's think here... hmm... what the name of this thread:
100% disk access and 100% RAM usage for no reason

No, sense in trying to help here. Ugh...

 

[aaronfross]

Seriously, how could throttling the CPU help prevent excessive disk or RAM access?

[aaronfross]

There are other reasons for partitions. Actually I never even heard of the idea that they would improve performance.

Mainly, it's for convenience and time savings. Partitioning is helpful for me to organize my data. Splitting a 14GB platter drive into three partitions helps me keep my data in a logical framework. It also helps with backups. By keeping data in different partitions, I can, for example, back up active projects while leaving archived projects alone. This saves a lot of time, because the mirroring software I use needs to parse the entire job on both the source and target drives before it does any transfers. If it had to read all of the data on a monolithic partition, it would take way longer.

[R Neil Haugen]

I'm used to using folder structures for organizing like that. And none of the backup software I know of has any issue with abiding by folder directed work. I assume I'm not familiar with your backup software.

[aaronfross]

Honestly, the partitioning scheme I use is more for my own organizational process than anything else.

[MyerPj]

It's nothing more than a inflexible folder arrangement, which extra processing needed to manage the partitions. The HDD Head has to move back and forth from the partitions. And there is absolutely no advantage to backing up to a partition, if the HDD goes, all the partitions go. That totally defeats the purpose of backing up. You might as well just do it to a folder.

If you are partitioning to have a different OS for testing, or use otherwise, a case could be made. But HDD and SSD are pretty inexpensive these days so, pickup a new drive and use that for backup and even a different OS.

[DavidVClarke]

Rather than disc access i still things is related to the clips.  Have you seen this thread: https://community.adobe.com/t5/premiere-pro-discussions/cannot-import-an-avi-project/m-p/14717576#M5...

[aaronfross]

Thanks, I already tried updating the Blackmagic drivers and installing 2023.0. Maybe I also need to uninstall Premiere 2024.5? But I can't do that because I have active projects that require that version. Unlike Maya, I don't believe Adobe project files are forward compatibile across major numbered releases.

It just goes to show the deeply problematic nature of the entire paradigm of constant updates. Even if the updates aren't forced, they introduce all manner of bugs and incompatibilities. The only solution to those is more updates. Which introduce new issues that require more updates. Gone are the days of a software application being released as free of bugs as possible. Stability is a thing of the past.

Then add draconian policies around subscription models, and it's a recipe for disaster. We're not even permitted to run any version of Creative Cloud subscription apps more than two years old.

All of this is decidedly hostile to getting any real production work done. Forget about revisiting an old project. It's very likely to be inaccessible. And with a complex project, I don't even know how we can work around that. Archiving is a disaster. We might as well go back to film and tape; many of the benefits of nonlinear editing have been negated by these horrible corporate policies and practices.

[R Neil Haugen]

There are several million daily users of Premiere Pro, many for broadcast/streaming and corporate web work. They are working without issue. So those with problems are not the vast majority, and that's something a lot of users struggle with understanding, That said ...

First, there shouldn't be any issue with multiple major versions installed. Most of us long-time users advocate never uninstalling a version. This machine has back through the 2019 versions installed, PrPro, Ae, Au, Me, and others. My old (now 'backup') machine has clear back to the original CC and CC2014 versions ... including SpeedGrade, alas ...and yes, I occasionally do take something into SpeedGrade for specific uses.

And I've never had an issue with that practice.

As to the comment that "we're not allowed to run versions more than two years old" ... balderdash. You can run any version you have installed. I and many others do routinely.

Next, subscription or not isn't actually an issue either. I've got software from both subscription and purchase, and they all can and do have bugs. Was it easier years ago to produce software with fewer bugs? Oh Hades, yes.

There were far fewer pieces, makes, and mods of hardware and OS out there back then. For another example, as BlackMagic has been working at turning Resolve into an Adobe clone, "one ecosystem for all video post" ... they've been having more bugs and odd behaviors across their user base. And yes, I work daily in Resolve, and am also active over on the BM forums, plus I teach pro colorists via the MixingLight website ... mostly based in Resolve.

Resolve is simply a loss-leader for BlackMagic, it's a way to get us hooked into their system to buy their hardware. Like many others, I've got enough BM hardware to have several licenses in a drawer here. They ... don't make it easy, at all, to use non-BM hardware with Resolve. Not because they're evil or greedy, but because that's how they make a living.

Subscription is the same way. It's neither good nor bad, simply one model to make a living for those employed by a company.

The only pretty much bug-free video post app I know of is Baselight. How do they do it? Old-school.

You buy the computer from them, with all allowed software installed, for between $14,000 and $20,000. For the computer alone. You NEVER EVER add another app or utility to that computer.

Next, you pay a couple thou a year for a license to use the app on that computer. And yes, for that you get an awesome app that is rock-solid stable. Of course, if you need to go setup in another studio for a week, you must dismantle and pack your system, move it, and set it up there.

Companies are simply groups of humans. Humans by nature are both incredibly variable and freaking all over the place. No two groups will ever come to the same list as "the important things!" let alone design the same setup. I have talked with a lot of Adobe supers and devs, and well, many of my 'big things!' are bluntly going to be used by too few people to have a chance at being incorporated. Yet a few have been.

Same with BM, although I don't personally know any staffers there. I've talked with enough of them at NAB/Vegas for a decade to hear pretty much the same comment. Ah well, right?

And understand, I understand the frustrations using any and all of these applications. Ain't none of them perfect. So I've learned to adopt a very practicality based attitude ... what gets the work out the door, works. Do that, whether I emotionally like it or not. Keep the emotions out of it or you'll go nuts.

And use whichever of the Fancy Hammers works ... as that is all any of these apps are, fancy hammers. I don't waste a moment of emotions on a hammer.

[aaronfross]

I cannot download any version of Premiere, or any other major CC subscription app, that is more than two years old. Unless there is some super secret archive of old installers. Make with the links.

If I had a time machine, then I would go back and tell myself not to uninstall anything. But then what happens when this machine dies or needs its OS wiped?

Corporate policies are getting in the way here. No question about it.

Peace out

[Alex367466572b5i]

There is also a problem with DV AVI videos...

[Kevin-Monahan]

Hello @aaronfross,

Thanks for writing in with your bug report. If you can provide further details about your setup, the team would appreciate it. See, How do I write a bug report?

Has this workflow worked in the past? I read about another user running the files through GSPOT and found that there was an audio codec error. See this post: https://adobe.ly/3WhU3Sr

Is it possible to use a different codec that might be less problematic for your workflow? Let the community know.

Sorry for the hassle.

Thanks,
Kevin

[Kevin-Monahan]

I have updated the status of this bug, @aaronfross.

Thanks,
Kevin

[Alex367466572b5i]

@Kevin-Monahan 
A video showing this error
_https://vk.com/video-14495031_456243037

[aaronfross]

Thank you Kevin,

It is not my responsibility to find an alternate workflow for a bug that your software introduced in recent versions. It may be a practical necessity, but I resent the suggestion that the burden of dealing with this lies on the user.

I do not need to run an analysis tool to tell you how the Blackmagic MJPEG codec works with Premiere. Or, more precisely, how it is supposed to work. The Blackmagic MJPEG video will load natively in Premiere without any issues, although Premiere interprets the video as full range when it is in fact limited range. (Fixable with Lumetri preset.) The audio will not load in Premiere unless the Blackmagic Desktop Software (including drivers) is installed. This is the way it has always been. And that is what the thread you linked is about. If the Blackmagic codec is not installed, the system will not recognize the audio. This is not an error per se, but simply a missing codec. Yes, Blackmagic messed up here, they could have just used uncompressed PCM, but they did something weird. Maybe it's a cross-platform issue.

This setup worked fine in Premiere 2022.x. I have experienced massive performance issues with every version of Premiere I have tried to use since then. And since I am not permitted to download v2022 anymore, I cannot revert to a working version. My project file is useless. It will not load or play video at all. And I have no recourse. So you can imagine my infinite frustration.

Here are my device specs:

Processor Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz 3.41 GHz
RAM 32.0 GB
Edition Windows 10 Pro Version 22H

NVIDIA GeForce 3060, driver 555.99
System drive: Samsung SSD 850 EVO M.2
Data drive (where video files are stored): WDC WUH721414ALE6L4 SATA

[MyerPj]

FWIW: Your system is a little weaker than the one I built in 2017. I had a i7-v6850 and 64mb of ram. The 850 evo is a SATA drive a standard SSD, it's not an M.2 and your data/clips are stored on a spinning drive.

That's not really a system that you can keep doing upgrades to software into 2024. At some place the software overwhelms the hardware. If your's kept going to 2022, that's pretty good. 

I replaced that machine in 2022, with a 12th gen i9-12900k and that's working well currently, and has 4 M.2 drives.

[R Neil Haugen]

That's a nine year old 4 core CPU ... it's totally ancient by modern standards. It's surprising Premiere runs at all on that CPU. No wonder it's maxed out all the time.

Our first computer for our business was around 1987 or so ... an 80386 we got, with the added RAM to have a full ridiculous amount of RAM, a full meg of RAM! Our pro peers laughed at it. And ... we got the truly massive 40 meg hard drive! Absurd! You'll never need all that!

I don't know how many computers we've had since. Many. I've had ... five laptops, alone ... maybe six actually. My "new" rig is a 3 year old 24 core Ryzen 3960x with 128Gb of RAM and a 2080Ti. That GPU I'll probably replace with a 4070 this fall. Or something like that.

I get 4-five years for a "main" computer, which when replaced, is my backup rig. I hate migrating to a new computer, but it's absolutely necessary to keep working away.

Tis Life in the Computer Age.

[Alex367466572b5i]

CS6 is working fine, Resolve is fine, and does 2024 necessarily need a PCIe NVMe drive for DV AVI?
And for 16K videos, what is needed then?
There is no need to invent anything, there have been such bugs before...

[Christian.Z]

this happened with me several times it was either a corrupt file or an essential graphics template that was screwing everything.
if you had an essential graphics tem p,ate that was built for a previous version of premiere, that might be your issue. otherwise, check on which file the conforming is being stuck and it will give you a hint if you have a corrupt file

[Alex367466572b5i]

Watch the video above... HDD for some reason loads video into RAM, SATA SSD does not behave like that anymore... PCI e NVMe is fine.
Just import video, DV AVI... CS6 is fine, Resolve is fine... 2024 is fine with PCIe NVMe only.

[R Neil Haugen]

I'm always amazed at the difference between machines. You say Resolve is fine on that computer.

I've got a laptop that is from early 2019, and was a powerhouse at the time. Specs that blow your rig away. And whole it ran R17 ... OK... Resolve 18 was such a dog on it I un-installed it. But it runs 24.x Premiere, both shipping and public beta, just fine.