Any way to prevent dupes in Premiere Project file?

[RyanPatch]

Due to working with both PluralEyes as well as multiple editors on the same project, I deal with importing a lot of XML and PRPROJ files into existing project files that I've worked with.

This, on a large scale, becomes untenable because every time I import another editor's work or a synced sequence, I get duplicates of all my media in the project. I'm trying to figure out if there any way to have Premiere check to see if the media is already imported, and reference that media in the imported sequence? If not, I will just keep getting larger and larger project files with multiple references to the same media, because both me and my other editors are working with the same media, but for different sequences.

As far as I can tell, when you import an XML, or even a sequence from another Premiere project, it creates new master clips for every clip used in the imported sequence. It does not recognize that you may already have a master clip that you're using in another sequence that is referring to the same media.

So if you want to keep both sequences in your project, you need to keep both master clips. If you delete a master clip from your bin, it will not stay in the sequence(s) that refer to it, it will get cut out, leaving a big steaming crater where it once was. Even if the sequence is not currently open. This is one of the things I loved about FCP7. You could delete every single one of your master clips and your sequence would be totally unaffected. You could even recreate the master clips by dragging them from your sequence to the bin.

I think if there really is no way to manage these duplicates, this is a HUGE problem for professionals who are working in environments with multiple editors. This isn't just a "well, learn how to deal with a new editing system" - this is actually a deal-breaker; and actually the only one that I see REALLY preventing Premiere from being the go-to choice for larger post houses. This problem becomes so big so fast that it makes true collaborative editing downright impossible. In my office we might have three people on a project, all editing and revising segments and passing them back and forth. On FCP7 this was easy as pie - we'd just cut and paste between project files and use basic versioning best practices. In Premiere, our project files quickly become nightmares and work is often inadvertently deleted or lost.

I would like to see:

- Smart media handling when importing sequences and projects. Premiere should look at the filenames and file location and attempt to relink any duplicate media. If it stumbles, it should ask for help like FCP.

- A media consolidation inspection feature. I'd love to see an option for inspecting your project for duplicate media references. When found, Premiere should automatically consolidate.

- Streamlined sequence exporting. You should be able to export a single sequence. I know there's some version of this in Project Manager, but we all know it should be easier than this!

Anyone have any ideas on how to fix this?

[Kevin-Monahan]

Hi Ryan,

many times some clips just dupe when they come in for no reason.

It's usually because some editor in the chain did not adhere to the way Premiere Pro likes projects to be organized and edited. Most issues are around ingest, sorry to say.

Thanks,
Kevin

[Darius Family]

It's usually because some editor in the chain did not adhere to the way Premiere Pro likes projects to be organized and edited. Most issues are around ingest, sorry to say.

Is there documentation that goes into detail about 'how premiere likes projects to be organised'? So we can all know the correct way to organise the project so as to minimise issues?

This makes for a good solid pipeline for a Read/Write project taking in Read only projects—the way Premiere is intended to work.

Can you elaborate on what you mean by read/write projects? Are you referring to when you open a project via Media Browser and the timelines are read only?

Thanks Kevin, I (and i'm sure everyone else) really appreciate the time you're taking to reply to this thread. It's such an important topic, for us. Clearly we're all really keen to be able to nail our workflows to avoid inefficiencies like duplicate clips. We just need all the right information to make this a reality (eg XMP ID's, RW v R project etc).

It seems there isn't a definitive resource on how to achieve this. I've seen videos talking about it (like the one you linked earlier), but i think we all really need an indepth KB article that sets out all of the rules that we need to follow to ensure our workflow/pipelines are in line with Premiere Pro's 'best operating practices'

[Crisis892000]

It's usually because some editor in the chain did not adhere to the way Premiere Pro likes projects to be organized and edited. Most issues are around ingest, sorry to say.

Kevin, again, thanks for all of your responses to this thread, and for trying to help us all out. I hope this issue can be resolved, but regardless, you have been a rockstar.

To respond to your quote, even if we did all adopt Premiere and everyone agreed to do things the proper way, I work in a department with 12 editors and 4 AEs. Sometimes people just forget, and if one person in that pipeline forgets to check an option, there should still be some sort of cleanup tool to fix this problem.. Someone WILL eventually forget. And there needs to be a remedy for it.

Some editors just have habits, good and bad. Some of them have ways they like to work, and my company feels that if they get their work done, why put in unnecessary restrictions? And that's the reason that out of twelve editors, only two of us have adopted Premiere. The rest of them are still on FCP 7 because they find is less restrictve.

And this issue may go beyond one or two solutions. There are many small tools I think we could implement that would alleviate this problem. As I mentioned above, a "Rename Clip to Match Filename" alone would solve many headaches.

[Kevin-Monahan]

Hi,

Please make a feature request to "remove duplicated clips when sharing projects." I think the engineering team may be interested in doing this should there be enough interest. It certainly sounds like that's what is needed.

Thanks,

Kevin

[mjshade]

I appreciate your attention to this conversation, Kevin.
I have submitted a feature request. I'm still encountering this very frustrating and limiting problem. Just wanted to chime in that it's still an issue in hopes that it won't fall off the radar.

[tp-doku]

4 weeks later - still no solution for this main problem?

As Darius Family said: "we all really need an indepth KB article that sets out all of the rules that we need to follow to ensure our workflow/pipelines are in line with Premiere Pro's 'best operating practices'"

We are working since 3 years on a 4K-long-term-documentary with lots of different sequences and we really love the possibility, that Premiere can handle native XAVC, but the "Duplicate-Media-During-Project-Import-Bug"is really something that drives all of us crazy.

[Kevin-Monahan]

Hi tp-doku,

4 weeks later - still no solution for this main problem?

Bug fixes and feature requests along these lines may require more patience. My apologies. Have you filed a feature request for your issues yet? If so, thanks.

"we all really need an indepth KB article that sets out all of the rules that we need to follow to ensure our workflow/pipelines are in line with Premiere Pro's 'best operating practices'"

That would be my responsibility. Sorry that I have not had time to put together a document yet. I will do my best, however, most of the findings of such a doc are contained in this thread. I'm not withholding any information, however, may need to gather more info which may take a little time. Sorry for the delay.

The "Duplicate-Media-During-Project-Import-Bug"is really something that drives all of us crazy.

Yes, you can avoid that if you are the project owner and the users ingest and share projects with you properly. For any stray clips, I hope that a new feature can help "clean" your project. Again, thanks for your request for that.

Thanks,
Kevin

[sr123456789]

Wondering when this is going to be fixed. I can't believe this box in question isn't checked by default. Or there isn't a warning of some sort. How many people have to learn the hard way?

I have cut a bunch of sequences already, some of which have been passed between computers, and therefore duplicates files are showing and the project size is growing. I am trying to find a way to clean up the project. I have gone back to an older version of the project where there were no duplicates and then attempt to import my sequences (from current project) into that "clean" project with just the footage loaded. Without fail it creates duplicate files. Even if I first make my current sequences offline before importing via media browser.

I tried starting a brand new project, check the box in preferences-->Media-->"Write XMP ID upon import..", then import the footage from the "clean" project, and then import sequences from current project, I still get duplicates.

Please advise if anyone has any insight.

Many thanks!

S

[SquareEyz]

Thank you (all) for your time illustrating this ongoing and frustrating issue.

Chrisis892000, Do you mind sharing your current workflow?

I'm using CC 2017.1.2, and its still a total pain that I have not been able to resolve.

Many thanks,

David

[Kevin-Monahan]

Hi Darius,

Please for pete's sake, put this information into a KB article somewhere so that others can know!

Will do! I agree that it's needed.

Thanks,
Kevin

[HenryRsL]

Hi Kevin thanks for your workflow.

I have tried it several times, but Premiere still generates duplicates of my footage.

The weird thing is that it works with some (just a few) sequences though.

To explain I have 8 different projects with approx. 5 sequences per project to migrate.

Does anyone have any other suggestions?

Thank you!

D

[Kevin-Monahan]

Hi GotzD,

I have tried it several times, but Premiere still generates duplicates of my footage.

Please describe the exact scenarios in which you still receive duplicates of your footage?

Thanks,

Kevin

[nellHELL]

Yeah - My problem too

Kevin-Monahan​  all of these dupe problems inexplicably work or don't work toggling between intended settings.

I can have "allow for dupe media" checked and it doesn't dupe, and unchecked and it does.  

Inconsistent and still a problem.

And doesn't checking the "Write XMP" box severely slow down the speed of the program? 

I was warned to uncheck it if I have nice timecode embedded  dailies -- what's the low down on that?

[hannahh59478303]

This is the perfect solution for importing synced xml's from PluralEyes 3.5 into Premiere w/o duplicating footage within the project, however my audio files are still duplicating when I bring the synced xml back into the project. Does the  "Write XMP ID to files on Import" preference not apply to audio (.wav) files? If it doesn't is there another workaround that would avoid duplication of audio files?

Thanks in advance for your response. This ability will really help in my feature doc workflow.

[Kevin-Monahan]

Hi Hannah,

I don't see any bugs written against your workflow. I can look into a possible workflow or workaround when I return to the office on Monday. Please send me a PM early next week and I will ask around to see if anyone knows of a potential solution for you.

Thank you,

Kevin

[Trelawney57]

That's good to know Kevin. We have had a similar issue...Thanks

[monkeychicken]

I think I found a solution - similar to post #46 by SoFreshFilms

(which didn't work for me)

1. Open source project A from offsite collaborator (who has identical media on a different drive)

2. Reconnect Media

3. Save and close project

4. Open destination project B and import sequence from project A (either by file->import or media browser).

No duplicate clips! Once Premiere knows where the media is, it doesn't attempt to create new clips in the project panel. At least it looks that way in the tests I've done so far...

Hope this helps.

[R Neil Haugen]

One of the bugaboos I've just learned about is that rather than importing the PROJECT from a collaborator working on the same project ... one should use the media browser to open the project and import ONLY the sequences. Then re-connect them to media as needed. Did not know of that step ... which is a game-changer.

Neil

[monkeychicken]

Yeah, that's what I've been doing but Premiere creates duplicate clips unless I open the source project first and reconnect it to my media and save it. (The source is coming from another editor with a copy of the media on a different drive).

Matt

[Kevin-Monahan]

Hello Monkey Chicken,

Before beginning your project did you do the following?

• In Preferences > Media, enable “Write XMP ID to files on Import.”
• This tags each media object with a unique ID, which Premiere Pro can track for items like duplicate clips.
• If this is not done, you will get duplicate clips throughout your project when importing sequences from other editors.

Thanks,

Kevin

[monkeychicken]

Unfortunately we started the project before I knew about the "Write XMP ID to files on Import" preference

But I'm hopeful about this workaround....

[Kevin-Monahan]

Hi monkey chicken,

I will ask if there's a workaround for those that forgot to enable the preference before beginning the project. That one thing can cause a lot of issues for those that had not idea about the preference.

Thanks,
Kevin

[monkeychicken]

Hello,

Just to follow up - so far so good with my workaround above. Have sent sequences to 2 different editors already this week and no duplicate clips!

Matt

[R Neil Haugen]

Sounds good ... keep us posted if this does work continually. Useful knowledge ...

Neil

[RyanPatch]

It seems like"Write XMP ID upon import,,,:" should be default selected, don't you all?

R