Media Encoder using too much memory

[Scott5EA4]

Not sure how to resolve this, but ME takes up gobs of memory during the encoding process no matter what I seem to do to alleviate it in Premiere. You can see screenshots below (but total memory was up to 98.63 GB when it finished). 

In general, I've been trying to export a 1-hour long video. Originally shot on iPhone (.mov) and encoding to MP4. My system would either crash after an hour or so of encoding. I think converted the MOVs to MP4s and tried that approach before encoding, but same issue. I had to cut the video to 30 minutes just to get an export to work. There has to be a way to improve the memory usage on my system for this task.

I'm running an M2 Mac, 24 GB, Ventura (can't upgrade just yet). I had all applications closed and only using ME. I was also using Hardware encoding (Metal). I tried Software encoding but it crashed on that as well (and 10 times longer to encode). I even tried medium bitrate but didn't make much difference.

It seems like ME is super inefficient with memory use. I know there must be something else that can be done because I'm clearly not the only person trying to create 1-hour content. Thoughts?

[EckiAME]

I will try to repro. Thanks for your feedback and sorry for your issues.

[Scott5EA4]

Thank you. Please let me know if you're able to reproduce and if there's any additional settings that may work to decrease the amount of RAM being used.

[EckiAME]

Did you record in ProRes or H.264/H.265?

[Scott5EA4]

MPEG-4 AAC, Timed Metadata, HEVC

I recorded audio separately, however, and that was 4-channel, 96000 WAV using a Zoom 360 H3-VR. 

I've also set my sequence to match and have exported to a format that will allow 96...same issue. I've changed my sequence to 48 and export to MP4 as 48 and same issues. 

[EckiAME]

It is totally possible that the export crash is due to the audio part. I guess you need to set the audio settings for the sequence correctly. Did you try to export the iPhone footage and the audio tracks separately to see what causes the crash?

[Scott5EA4]

No, I didn't try exporting them separately. Let me try that and see what happens and I'll respond.

[Scott5EA4]

I exported a 20 minute version of the video, and then just the 20 minutes of audio. The video was exported from MOV to H.264 and the WAV was exported to MP3. 

The video exported first and took about 30 minutes and consumed 40GB of memory. After the audio was exported, the total memory usage was at 53 GB (both ran sequentially). 

[Scott5EA4]

I should've said that based on that it looks like the video is the culprit, not the audio. BTW, you can see the video I'm referring to at https://www.youtube.com/watch?v=fR47e-3tVR8 

[EckiAME]

One important thing: do you send over the iPhone recording from Premiere Pro or do the transcode directly in AME? What is the memory consumption like after the encoding finished?

[Scott5EA4]

I'm editing in Premier Pro and send it to AME for the encoding. After encoding is finished the memory consumption stays the same...it doesn't go away until I quite AME. 

[EckiAME]

Do you use any mogrts, effects, filters, plug-ins, EssentialSound etc in your project? Do you use "Import sequences natively"?

[Scott5EA4]

I applied the Video dissolve transitions on the end/beginning, and in between the repeat of the same clip. I also had a third-party color effect added to the clip. The clip was about 3 minutes and just repeated for 20 minutes. However, even without the extra colorizing, I still experience the same memory issues. In general I create a new sequence and grab the video through the Media Browser in Premiere and add it to the sequence. For the audio, I cleaned it up in Audition first, then pulled in through media browser and added it to the sequence. Pretty basic

[EckiAME]

Using Adobe Premiere Pro 24.2.1.2 and Adobe Media Encoder 24.2.1.2 stable releases, I was not able to repro memory issues.

With a self-created iPhone footage project, it peaked at around 17 GB memory and got back to almost normal after the eoncde with "Import sequences natively" turned off.

When turning the option on, and resending the project which basically transcodes an MOV-based iPhone footage sequence to MP4, memory consumption went up a bit and did not go back to normal. Memeory remained at over 3,1 GB after the encode.

In memory settings in the AME preferences, of the 64 GB memory, 12 GB were reserved for other applications.

The project uses iPhone 14 and iPhone 15 footage in MOV format and Cross-Dissolve transitions. I copypasted the clips in the Premiere Pro timeline until I got roughly one hour.

So my question to you would be:

• Can you verify your memory settings in AME and post a screenshot?
• Can you retry without editing the audio and and external color plugin applied?

[Scott5EA4]

Sure enough, I had import native sequences enabled. I disabled that in AME and verified that memory allocation on my system was set to 18GB of the 24GB available. I started running my 20 min video again (no changes) and got about 30 minutes into it before I had to stop because of other priorities...but it appeared to be peaking around 36 GB of memory (including swap) for the PProHeadless process. AME was only about 800 MB. It was steady around the 36 GB for about 10 minutes. I suspect that will stay the same. I think unchecking that box in AME will do the trick. I'll run a full test when I get the chance later today, and report back later for sure. Thank you!

[Scott5EA4]

I ran the project again with, and without, the lumetric effects and sound edits. The import native sequences was disabled in both cases. 

Without any changes to the project, my 20 min video took ~59 minutes to complete, PProHeadless maxed at 79.12 GB memory; AME at 740.2 MB of memory. 

With the removal of the effects and edited sound, my 20 minutes video took ~27 min to complete, PProHeadless maxed at 70.43 GB memory; AME 768.6 MB of memory. 

Without the effects/edit the job was completed in half the time, but the amount of memory used was roughly the same (8 GB less without the effects/edits). My original video was 30 min when I started this thread and the one I've been testing with is 20 minutes. Overall it doesn't seem to be that big of a difference between checking/unchecking the native sequences box. If my video was 10 min longer, I could easily see it going from 79 GB to 98 GB (similar to the original issue).

At the end of it all, I think this is simply going to boil down to Media Encoder requiring a better system than what I have for anything longer than 20-25 minutes of video.

With effects

Without effects

[EckiAME]

Would you be able and willing to share your project and media for us to look into your issue? I can private message you the instructions in this forum.

[Scott5EA4]

Sure thing. Let me know

[EckiAME]

Please check your inbox in this forum.

[EckiAME]

@Scott5EA4 Do you also see this in direct export? It seems to be very slow out of Premiere Pro, too on my MacBook Pro 16" (2021, M2 Max).

[Scott5EA4]

Yes. Even on Direct Export from PP, I was still getting the increased memory usage to a point where it maxes out, especially if I'm trying to export something longer than 20-30 minutes. 

[EckiAME]

"Video" sequence did render for me using this configuration: MacBook Pro16" (2021, Apple M1 Max, 64 GB RAM) with macOS Sonoma 14.4, Adobe Premiere Pro 14.2.1.2 and Adobe Media Encoder 14.2.1.2. Memory did not get back to normal. It stayed at 26 GB. I will file a bug. Please note that we already started looking into thesem memory issues. I did not encounhter any crasher, though.

[Scott5EA4]

Thank you. It's good to know that we're not all losing our minds. It would be good to have the memory issues resolved so we don't have to worry about it spiraling out of control. 👍🏻

[Pericles.Badger]

I'm having the same issue here, I'm not running on an M2 mac tho, I'm on a Ryzen 9 5950 with 32 GB of ram and a 16GB 3060.

AME just stops at the end of the encode eating up memory and doing little else.

Hitting pause on the encode and resuming a few seconds later sometimes bypasses the issue. But this is getting bothersome.

This is my resource usage. 

And just as I was writing this paragraph the thing finally finished. 48 minutes after it got stuck.

The sequence being rendered is 56 minutes long and the final video is 15.8GB No effects, no filters just crossfades between cuts.

Any advice on AME hanging for 48 minutes after a simple encode?

[EckiAME]

Did you try to temporarily disable the antimalware just to see if it finishes then? We had a couple of issues with AntiVirus software blocking our write access.