Why does Premiere spend so much time "relinking media" when opening a project?

[Ovidius9000]

When I open a large-ish project, the Progress Dashboard on Premiere spends minutes doing what it say is "relinking media".   What is it doing?  Why is this necessary?  It is technically a "background" process, but nothing really works until it is finished.

After a few mins, or 5-6 mins on really big project, it finishes "relinking" and then I can edit normally.  But my question is why can't Premiere just save the state it is when it's done and then just open quickly the next time?  Nothing has changed in my media files, why "relink"? 

NOTE: I am not talking about when Premiere cannot find the correct location and you have to point it to the right place. I'm talking about normal everyday automated stuff that happens when I open a project.

[Sumeet Kumar Choubey]

Hi Ovidius9000,

We're sorry about the poor experience. Where are the media files used in the project saved? Are these saved on an internal drive or external/network drives? Is this happening with any specific project or all of them? We're here to help, just need more info.

Thanks,

Sumeet

[Ovidius9000]

External drive.  It's about 50TB of footage. 

Why would it mattter if it's internal or external?  In 25 years working in editing and sound-for-film I have literally never been asked if something is on an "internal drive".   In professional settings we all use external drives so projects can be easily moved to a new system, or across the country etc, to fit the needs of the production.

[Sterphy]

It does a really terrible job relinking as well. I will specifically tell it which folder to search, but it will still search the entire C drive. 

[R Neil Haugen]

I'm surprised you've never had issues with external drives. Yes, they are heavily used. And for many workflows, absolutely necessary. But they come with problems inherent in being external. (And this doesn't include external RAID arrays connected via specific hardware bits.)

1) Data transfer is just never as good as an internal "directly on the motherboard" connection drive. USB and Thunderbolt sustained speeds don't match that with internal SSDs.

2) Drive names are an issue ... sometimes drives have a 'name' that is recognized and used by the OS, but often they get assigned a drive name when connected or the computer is booted again.  And that can throw things for a loop.

3) Premiere runs totally on metadata ... that's all Project files are, massive bits of metadata and text information. And if it needs to re-sort the drive name/files used in that project, it will have to do so for every file used. And that process involves replacing the last stored data with the 'new' data for the file, all written in the Project file's metadata ... stored between RAM/cache and data files.

And if you have preview files and such ... as is probable ... it has multiple files per clip to reestablish the link to.

So if Premiere is having to deal with a newly assigned drive name at launch, it will take some time to get all that sorted. 

BUT ... if  ... the drive has a name that the OS recognizes and uses on launch, much of that time is not necessary.

Now, there is a separate issue where Premiere doesn't remember it already has preview files and such, and recreates those every launch of the project. THAT ... is very annoying, but again, not the same issue as the above.

[Ross3888292368fh]

The drive is not changing, the media is not changing.  My question is: what is it doing when it says "relinking" everytime i open the project.

Other NLEs have no problem working off external drives.  Metadata is tiny in terms of bandwidth.  My question, again, is why/what is Premiere doing every single time the project starts up?  And why can't it save it's "post linking" state, you know what I mean?

[R Neil Haugen]

I answered that above.

If Premiere sees any part of the "name" as different, it has to rewrite all the internal Premiere metadata to that new name. Because your project is nothing but metadata.

So, as I said, it's not rewriting the original files. It's having to rebuild it's own internal directions ... (file-name locations) to all the original files/previews/whatever. And if you have hundreds of files, that can take awhile.

[Ross3888292368fh]

But nothing has changed, I already said that.  So can you give me any other reason it would be rescaning my media drives?  Nothing has changed in the folder struture/files for weeks.

[Averdahl]

That sounds like a corrupted Media Cache Database.

Hold down the Alt-key while launching Premiere Pro to get the Reset options dialog. Check Clear media cache files and click Continue. Open the project and do what you have to do and save the project and close Premiere Pro. Launch Premiere Pro again and see if it will promt you to relink anything and report back.

But nothing has changed, I already said that.  So can you give me any other reason it would be rescaning my media drives?  Nothing has changed in the folder struture/files for weeks.

By @Ross3888292368fh

[Ross3888292368fh]

The "Media Cache files" means what in this instance?   Does that mean all the pek files, etc?  The ones next to the original media files?   My worry is that with 55TB of footage, it might take forever to recreate all the cache files.  

[Averdahl]

I did a test and all peak files were deleted as well. So, all files in the folder Media cache and Peak Files will be deleted. I never store pek/cfa files next to originals, i always store them on an internal disk. But i use internal disks for all my video work so i don´t know how it works when deleting Media Cache when having that option checked.

But you can delete the Media Cahce folder manually in a file browser and thus leave all pek files untouched. Go to Edit > Preferences > Media Cache to find out were your folders are located if you don´t know. Make sure that Premiere Pro is closed when doing it. The folder named Media Cache is the one you want to delete.

Whopping 55 TB footage in one project or 55 TB in many projects?

The "Media Cache files" means what in this instance?   Does that mean all the pek files, etc?  The ones next to the original media files?   My worry is that with 55TB of footage, it might take forever to recreate all the cache files.  

By @Ross3888292368fh

[R Neil Haugen]

Averdahl has a very good suggestion, of course.

But whether or not you have changed anything, isn't the point. It's whether between the way the OS sees things and Premiere that matters here. And if the OS assigns anything different in the entire file tree thing, then Premiere has to rewrite all file linkages in its internal meta setup.

Normally, with external drives, if the official drive name is the same as the OS sees, and shows it as, then most often the connections stay linked. But there can be hiccups. I've had them on my much smaller projects at times.

Those hiccups don't normally occur with internal mouted drives.

I work for/with/teach pro colorists, who are always working from client-supplied files. And most of them have large hardware RAID arrays attached to their system, and they load all the client supplied files onto those hardware RAID arrays for working in either Resolve or Baselight or whatever.

As that is vastly more reliable, stable, and predictable, than simply attaching the client's supplied drive/s.

And that has nothing whatsoever to do with Premiere, note.

[Ross3888292368fh]

I don't understand how you can say it "isn't Premiere" when other software can work with external drives quite fine.  I literally have been working on feature size productions my entire life (Avid, LightWorks, ProTools,  and even something really old called Sonic Solutions).  Only Premiere seems so touchy and finicky.   

Using internal drives is not an option on any reasonably sized production, so whatever problem Premiere has with external drives should be fixed.      You can't just blame the OS –– Premiere could write their software to ignore OS changes.     That's what other people have done.

It's really a bad system to always rescan thousands of files.   There are so many ways to design around this.

[JOHN MONDO]

For some sanity, I worked with Avid for over 15 years and I can assure you it is a much worse experience relinking files in Avid. Yes, it's a very solid database but the way Avid links files is not automated at all, particularly if the media is in a container (eg older formats like XDCam) - everything has to be relinked manually.  Avid was originally designed to ingest camera based media into it's own proprietary MXF format - if you do that it's flawless but it takes up a lot of space - if you work directly of your camera media via linking I can assure you it is a much worse experience than Premiere.  However most Avid houses just don't work that way - they ingest everything to MXF via a dedicated hardware peripheral, generally SAN based (or at least they used to). 

Back to Premiere - I have seen issues before where once you move media once, the project will continue to want to relink those files everytime you open the project. I think the only way around this is to start a new project and copy all your bins into the new project - I'm not sure if that will work but worth a try. There does seem to be a bug in the program that means once you move something once, PP fails to update it's internal links, particulalry if you are relinking to a network location.

[Ross3888292368fh]

It's not re-creating anything, not creating any files.  It's just like scanning the whole drive.

[Averdahl]

Define "It´s". 🙂

When you write "It's just like scanning the whole drive" is "It´s" Premiere Pro or your OS that want to scan the drive?

I have never had Premiere Pro started or wanting to just scan my drive, but i have had the OS wanting to scan my drive to detect and correct disk errors.

The TS of this thread had issues with Premiere Pro relinking files, do you also have issues with relinking?

It's not re-creating anything, not creating any files.  It's just like scanning the whole drive.

By @Ross3888292368fh

[Ross3888292368fh]

When you open a lare project, there is a progress indicator in the upper right corner.  If you click on it, it will say "relinking media".   Adobe uses the word "relinking" in an odd way here so I say "scanning".   It knows where to look, it is fully automatic and takes place in the background, but slows down the system to a crawl for 5 to 10 mintues, depending how large the project is (how many clips are referenced).

My question is WHY does it "relink media" or scan the entire media folder, whatever you want to call it, every time Premiere is restarted.   Why can't it save whatever state it was in after it finished scanning.  This is the way other applications work.   Most other applications assume files are where they are the last time you opened the application and only scan if they run into problems.

So either Premiere just relink/scans the entire media folder every time it runs, OR it sees something as "missing" or needs checking.   I mean, that's how it seems to me.   

What furstrates me is on Premiere boards and support, nobody says "oh that's weird".  No, they say things like "are you using an external drive"?    Like, bro, it's 2024 and i work in professional post-production of course I'm using an external drive,  what kind of amateur hobbyist question is that?

I just can't believe Premiere doesn't save the info after it "relinks" all the media.  And if relinking is an error, I can't believe no one has pointed me to a log file or something where Premiere would log the reason why it descided it needs to look at the media folder again over and over every time it starts up.

[R Neil Haugen]

Read words.

It is re-writing the internal file linkage metadata, to have a proper reference link to each asset on that drive. That internal reference link is what all the bits of the project use to do anything to anything.

And it amazes me you insist nothing else has that problem. Because of a decade of time I've had just being around pro colorists who routinely have linkage issues in other apps.

Which as noted, is why they pay a ton of money to have massive external hardware RAID setups. Holding many TB of media.

And avoid external USB/Thunderbolt drives like the plague. And have endless discussions on which external RAID setup is the most cost effective at the moment.

[Ovidius9000]

I'm not haveing trouble with linkage.  I'm asking why Premiere has to rewrite the linkage every time it runs.

I mean a colorist has to RE-link to footage of course. I'm talkinga bout editing.  I'm an editor, not a colorist.  

So the answer is Premiere does have to rescan the ENTIRE media folder every time it starts up?

Litereally no one has answered the question.

[R Neil Haugen]

I have. You haven't accepted the answer. Rather simple.

Let's try a different phrasing.

Premiere has to have an absolute file name link to all bits used in a project. OK?

Premiere sees something that, to it, is any kind of change to the file name linkage data. Therefore it rewrites it's own sata files, to ensure using the file name data as it sees it at this time.

That difference can be down to even, say, Dropbox putting a "tag" in a file header. Whether it's about the last time it updated a file, or even scanned the file.

Which is not a change you or I would ever see. But is something Premiere is aware of. And therefore, as the file header data has changed, Premiere rewrites it's data accordingly.

[Ovidius9000]

Well, that SUCKS

[R Neil Haugen]

At times, yeah. Again, that's why all the colorists I know (and that's a lot, worldwide) avoid working from external "removable" drives whenever possible.

In-suite, they upload to their large external RAID setup, used only for client working files.

Now, when they're going "remote", working from laptops, yea, that's using USB/Thunderbolt externals. With all the potential pain that involves.

But do NOT have any file reading/cloud-sync apps set to work with the folder tree, on an external drive that you are using in Premiere.

[Ovidius9000]

I don't have anything syncing.  Like I said already, it hasn't changed in weeks.

I will just be sure never to work on Premiere again.  Contrary to what you're saying, other NLEs actually keep track of their media and only "relink/scan" when the app cannot find the media.  

I'm sorry that you feel this is "normal" computer behavior, but I can tell you that it is not.  It is just Adobe behavior.   I have worked with application designers and database companies.  None of them has ever said "oohhhhhh nooooo, not an eXterRNaL drive!?!"

[R Neil Haugen]

There are several million daily users of Premiere. Many of whom do use externals routinely. So it is done. If not recommended with large projects, especially.

You seem to be having more troubles with relinking than most folks, so I'm wondering what is different on your rig? Or setup? I'm not the local expert in drive file setups as far as the right data to put where. So I'm not someone to do much in the way of troubleshooting (with any expectation of success) how that specific drive is formatted and linked to by the OS.

That can make big differences. Oh, and certain characters in the file names cause issues also. Like puntuation type characters and whatnot. Or simply having too many characters in the full file name.

Can you explain why several hundred major colorists I know and of, most of whom never open an Adobe project, avoid externals like the plague they insist they are? And teach and share all their information on client data RAID array setups?

That is a very, very common topic on the BlackMagic forum, Lift Gamma Gain ... and MixingLight of course.

[Hanna Nicola29113413w6ve]

Hey @R Neil Haugen I do follow you. You're making sense. But. I have the same problem whilst using a very fast, internal NVME SSD for a large project (about 3TB). The project has a few clips from an external that it relinks. The clips relinked from the external do take way longer to comlete, but it still relinks footage from the internal NVME SSD.