Premiere Pro 14.3 - Generating new Peak Files for .R3D files every time I open the same project

[superneat12345]

This is absolutely absurd to me. I have a project with 26.5 TERABYTES of .R3D (RED Files) and Premiere Pro 14.3 (but the last 4 updates as well) generates new Peak Files for EVERY clip EVERY TIME I open the project. This process can take HOURS.

Why can't Premiere Pro refrence the peak files made for the exact same clip stored on my PC (As well as MAC from my tests) from yesterday? 

This process seems comically redundant and has been an issue since at least 2015 looking at the forums. 

Disabling automatic Waveform creation in Edit> Prefrences does nothing to solve or speed up the problem. It takes HOURS to load a project now. Which means I simply cannot shut the project down once it's loaded. 

ADOBE. PLEASE address this problem. 

[R Neil Haugen]

Where are the files stored?

Neil

[Shark Island]

This is also driving me absoultely nuts. I have a project whihc wants to generate over 22,000 new peak files and consequently takes hours. Have had the worst day today - achieved nothing. The number of these s peak files is likely to double over the course of the project as I am cutting an observational dococ for whihc there are often 8 tracks of sound for one piece of video.

Is the only solution really to leave the machine on all the time?? Not at all environmentally friendly.

Please help. I've wasted the whiole day. I only upgarded to latest version of PP last week and it is since then that this process seems to have slowed to snail's opace.

[R Neil Haugen]

Where are the files stored, both the original media and the peak files ... local disc, network server, what? Just trying to get more data here.

Neil

[stephenh71804362]

Hi Shark Island, has your PP system settled down at all? I have become quite adept at solving troubling issues with it... we have 4 offline rooms with various levels of cpu gpu etc ... in Sydney and happy to help. Is this Kate Hodges by any chance?

all the best,

Steve Hopes

[Michael Grenadier]

can you post a short clip and I'll see if I see the same behavior...

[salvo34]

hmmmm. I usually don't have that problem on a pc.

Typically I only do projects around 20 terabytes long from Red X and I always use the sound from 'in camera' like you because the quality is so good ( rather than slate and record from external recorder ).

I don't know why 26 terabytes would cause a problem.

Maybe you could try using an Arri camera instead ? Beats me.

[mstegner]

Hi, This is Matt from Adobe.

I can try to help but I'll need a little more information.  

What kind of NAS is the footage stored on?  

Was the footage imported via the Media Browser, or were the raw files dragged in from Windows Explorer?

Can you post pictures of a couple of your preferences so I have a sense of your configuration?  Can you post a picture of Preferences>Media Cache... and Preferences>Media... will be a good start.

"Disabling automatic Waveform creation in Edit> Preferences does nothing to solve or speed up the problem". This is one of the first problem solving steps that I would recommend.  If you turn off Peak file creation, you will need to close Premiere, and I HIGHLY recommend you manually delete the Media Cache Database, the Media Cache files, and the Peak files folders inside of the Common folder.  If there are left over .pek files that are still being generated , Premiere will attempt to keep recreating them even after you turn off the waveform generation.

Also, the media cache locations should be on a fast, local disk, not on a shared network disk.  Attempting to share the Media Cache among multiple users and machines will end in nothing but errors.  (Please understand that I say this with zero insight into how you have everything set up, and can only make broad statements about the functionality of the Media Cache)

[thxapproved2]

Matt,

I would like to post this here as well, as I would like to find a resolution to this issue:

I'm having this issue in all versions of PPro CC22 (Currently on 22.4). I've noticed it constantly re-generates the PEK files every time I open Premiere and then open a project. OF NOTE: It is only doing this to BRAW (Blackmagic Design raw files) and MOV files (h.264 or ProRes 422... may do others, but these are the only ones I am using).

Interestingly, it is not regenerating the peak files for AVCHD media (h.264 .mts files from Sony), MP4 media (either h.264 or h.265), WAV files, or MP3 files. This is machine agnostic potentially, at least as far as the Windows 64-bit version goes, as I have the same issue at home and at work on machines with different specifications.

I have tried all the suggestion of placing the metadata files in various places and clicking on and off the "automatic audio waveform generation" button. Nothing seems to make a difference. I even did a fresh install of PPro, along with resetting preferences.

Also, recently when importing media (after updating to 22.4), when I go to close Premiere, I get the "metadata writing in progress (xxx files)... if you exit now you will lose any unsaved data" warning. The xxx file number is identical to the number of files that have peak files being regenerated. However, my computer is idle. Any idea why it can't write metadata about these files?

Does anyone have any thoughts? It's not preventing me from working...yet. However it just means 30-minute documentaries take a long time to load and then be ready for work, as the system is very sluggish while regenerating 200+ peak files.

Windows 10 / 32GB / All local drives / not team project / have tried metadata files next to originals and in a media cache on another local drive. Makes no difference.

Thanks for any help. I have posted in the DVA Uservoice area as well, hoping to find an answer.

[mstegner]

@thxapproved2 Would you be be able/willing to get on a screen share with me at some point to try to resolve this?

[thxapproved2]

@mstegner I definitely would.

[Jeanette3336388335ws]

Hi, I am having the same problem. I have a 24 TB project (documentary), and it takes days to finish the peak files. I leave the machine working last night and it continued the process. Sometimes it gets stuck and fail, it simply quits. Every time I want to work, it takes about an hour to load all the project, then it continues to generate the ethernal peak files. If I want to show directly the film from the computer, the presentation will get stuck. We have mixed footage, RedCam Komodo (6K), Sony FX3 I think, iPhone videos, super 8. I don't know what to do. I have two Thunderbolt Lacie disks of 20 TB each, and a Crucial SSD where the caché and the peak files are located. I have deleted them several times, as the film and the edition stucks. I really don't know what to do, the director is desperate. I have tried this two computers, one M2 Mac Mini 8GB in RAM and a 2017 iMac 64GB RAM, and in both it does the same thing. I am using Premiere Pro 2023 (with the last update), and I made a copy using 2024 with the same results. I uploaded a small video so you can see how hard are working the disks. They are connected with a thunderbolt 4 cable. Please help, Adobe!!!

[Michael Grenadier]

just a couple of longshots (but might be at least part of the problem).   

Make sure in your preferences:  media  that you turn off write clip markers to xmp, and do not enable clip and xmp data linking.    It's possible that this makes Premiere think that peak files need to be regenerated...   I was using carboncopycloner to do incremental backups of my media drive...  It was backing up many files that shouldn't have been needing backukp.  When I turned these options off, the problem disappeared.  

The other thought:  Had a client who had long project loads using a pegasus raid.   The client had renamed folders with restricted characters, in this case / using dates (for example 2/3/24 Instead of 2_3_24, and it seems it was probably causing intermittent and unpredictable issues including extremely long load times..  When I insisted that we rename all the folders...  he fired me...   

[Jeanette3336388335ws]

Hi Michael, the XMP is turned off. I have lots and lots of markers and transcriptions as this is a documentary feature. Let me check the folders, but I loaded them without restricted characters as far as I remember.

Thank you very much 🙂 If you have any other thought, please let me know, we are driving crazy with this issue.

[Michael Grenadier]

OK.  Just throwing a few thoughts out.  

Make sure you have full permissions on the drive where your storing your peak files.   This is an OS setting.  If you need help with this post back.    And make sure your mac user account has administrator privileges...  Do any of your drives require software installation?  Just another thing to check.  

Are you turning off the computer at the end of the day?  Might be an idea to try and keep it on and see if that reduces the load times.  And maybe leave the Premiere and the project running overnight.  Should be able to put the computer to sleep to save energy without screwing something up.  

Wondering if any of your camera original has compressed audio?  Could cause issues with peak files (just guessing again)

Might be an idea to create a new projects and just bring in material from one camera for each project and see if any are requiring the long peak generation...  

Make sure you have sufficient empty space on all drives.  At least 10% on all media drives and 20% on your startup drive. 

This is something that should really be built in to the OS...   

Have you generated proxies?  Wondering if you could disconnect the fullrez media and just work with the proxies....  and then reconnect fullrez media when you've finished editing...  

Hope some of this helps.    

[Michael Grenadier]

Also, are you connected to a network?  Try pulling the ethernet cable or turning wi-fi off...  Just a troubleshooting step..

And are you running ANY other programs at the same time.   If so, try turning them off.  Creating a new User Account (in Mac system preferences) and see if that helps.    Besides log in items, apple also allows other programs to load in the background.    In system preferences, go to log in items and you'll see a whole bunch of stuff that are "allowed in background."  Software installation often adds these items...  Creating a new user should eliminate those, but always a good idea to check these...

And run diskutility:  disk first aid on all media drives...   and maybe even your startup drive by booting in ... damn having a senior moment.  There's a way to boot maybe holding down command-r which will allow you to run diskfirstaid on your startup drive...  

[Jeanette3336388335ws]

OK.  Just throwing a few thoughts out.   - Thank you! Make sure you have full permissions on the drive where your storing your peak files.   This is an OS setting.  If you need help with this post back.    And make sure your mac user account has administrator privileges...  Do any of your drives require software installation?  Just another thing to check.   - I have full permissions of the disks and we are working with my Mac Mini. Are you turning off the computer at the end of the day?  Might be an idea to try and keep it on and see if that reduces the load times.  And maybe leave the Premiere and the project running overnight.  Should be able to put the computer to sleep to save energy without screwing something up.   - Yes, that helps if I work in my house, but the problem is that everyday I have to move with my computer. Sometimes the disks are with me, and other times I leave them in the director's house, but the computer, as I use it in other projects, it's difficult to leave it there. 😞 Wondering if any of your camera original has compressed audio?  Could cause issues with peak files (just guessing again) Might be an idea to create a new projects and just bring in material from one camera for each project and see if any are requiring the long peak generation...   - I am not using the audio that is inside the video created from the camera, I use the audio recorded in a Sound Devices or a Tascam. All that audio is in WAV. - I have tried creating a new project, but my sequences are pretty long. I tried to copy one of the sequences and it took the Premiere to copy it about 5 or more hours (I went to sleep after the 5th hour). I may bring all the sequences one by one, and let's see if that works. Make sure you have sufficient empty space on all drives.  At least 10% on all media drives and 20% on your startup drive.  - I have the reglamentary 10% of empty space. My startup drive is complicated, I only have 250 gb of space and I am always fighting deleting the downloads. I usually have 50GB, I will try to erase more files and programs, and see if that helps 🙂 This is something that should really be built in to the OS...    Have you generated proxies?  Wondering if you could disconnect the fullrez media and just work with the proxies....  and then reconnect fullrez media when you've finished editing...   - I made the proxies directly from Premiere and Media Encoder. Then, I linked the proxies to the full res material to edit and have a high resolution file at the end of the editing. I want to disconnect the full res, but it only disconnects the proxy files. If nothing works, I will have to edit again using only the proxies and forget totally of the full res. I may do a new project, disconnect the full res, and linking again using only the proxies and see what happens. Hope some of this helps.     - Thank you Michael, I will check all the information again, but I think I will have to reconnect everything in a new project and leaving behind the full res, so we can finish editing. They will continue shooting in about a month, and the project will grow again, and we are still missing about 40 minutes of the film. Thank you !!!

Reply to this message

[Michael Grenadier]

even though you may not be using the audio from the camera original, if it's in the project, it may be generating peak files.  I'm not sure about this but maybe someone else who has a better handle on this can pipe up. 

As far as working with just the proxies, Maybe start a new project, duplicate and import  a small camera roll and proxies and then move the camera  original to a different location or just nest it inside a folder so Premiere can't see it and see if you can just work with the proxies.   this is definitely a usable workflow, just can't remember the details of how to make it work.   Or just relink the clips in Premiere to the proxies...   

Can't remember the reason, but I had to do this a few years ago...  took the camera original offline and then relinked to the proxies, and then when it was time to finish, relinked to the camera original.  I'll probably remember by morning.    I do remember that I had generated proxies with the same pixel dimensions as the camera original...    I remember now.  Sony had changed the audio format of one of their cameras, and it was impossible to create proxies within Premiere that would stay linked because you couldn't match the audio format even thought there were at most 2 channels of usable audio content on the camera original.    I did make this workflow work, from soup to nuts...  But always crucial to test the workflow before spending much time to get it to work...

[Jeanette3336388335ws]

Hi Michael, I will do what you say tomorrow, the computer continues generating peak files. I will leave it working all night and see what happens. The other option I am thinking is to make a media manager what I have edited and bring it to a new project, rename the old one. And do what you also say, move the raw material to another place and see if it relinks the project only with proxies. It will take long time to do it. Thank you very much 🙂 Good night!

[Michael Grenadier]

test the workflow with just a couple of files before doing everything..

[salvo34]

I contacted a DP who used to work on episodic TV stuff, and asked how many terabytes he used per episode, so he contacted an AC friend and got this info:

We shoot with 3 sony cameras and each one is about 1 terabyte per day... the shoot lasts 10 days per episode, so that is 30 terabytes.

I didn't get dimension or codec info, but for TV broadcast it's probably full HD most of the time. Some stuff might go up due to VFX down the line, but I  kinda doubt it. It's like a factory... the workflow.

This post is so ridiculous it's beyond silly. Good luck getting truth and specs, cause nobody in the world in pro work would have posted a whining complaint like this, blaming Adobe for the past 5 updates for this without adding all the info, you, Matt, just asked for.

[David Diaconescu]

Good day! Has this issue been solved by any chance? I would love to know what the root of this problem is because I am having it as well...

Thank you!