Solving Duplicate Master Clips AFTER they have been edited into timelines

[fierce_self0D45]

I'm editing using the most recent version of Premiere on the most recent version of OSX.

I'm working with two other editors who have the same versions of Premiere and OSX as me. We have been importing sequences/project files back and forth for several months, and we didn't realize that this was causing the master clips from each sequence to create duplicates of themselves upon each import from one project file on one editor's computer to another.

When we realized it was happening, it caused an organizational nightmare, and we didn't know how to fix it.

After reading through this guide: Reuse clips and work across multiple projects | Adobe Premiere Pro CC tutorials I think we've figured out how to prevent dupes going forward.

HOWEVER, we have a huge number of duplicate clips that we can't get rid of now. If we delete one dupe master clip, it deletes itself on whatever timeline it's been used with. We will lose our work if we delete any of our master clips.

My question is: is there a way to make Premiere realize that all of our duplicate master clips are the same media and winnow them down without deleting their appearances on our timelines?

THANK YOU

[Kevin-Monahan · Accepted solution]

Hi Mjshade,

is there a way to make Premiere realize that all of our duplicate master clips are the same media and winnow them down without deleting their appearances on our timelines?

I don't believe so. All these imported clips have different XMP IDs, so that Premiere Pro thinks that they are not related to each other. You need to enable writing XMP ID on import before you even begin the project so that Premiere Pro can track what's what. Otherwise, I think you'll have to live with this anomaly, which I agree, can get quite nasty.

I'm writing up a workflow doc to assist others in a "shared projects" workflow, but I'm afraid most of the time a situation like yours occurs where you get the info too little too late.

If anyone else has ideas, I'd like to hear them.

Thanks,

Kevin

[veniceedit]

How can we view and check an XMP ID to ensure it is the same between projects? I have a similar issue, but only half of the clips created duplicates, while the others linked just fine.

[Joe_G!]

Hi Kevin,

**Thanks ahead of time for your help!!**

We are running into an issue with duplicate clips and the source media is from a C300. The XMPs for the original clip and the duplicates are the same (via reveal in finder). We thought it was a spanned clip issue -- Premiere reads a spanned clip in timeline and brings in other half of file as a whole clip -- but this also happens to normal C300 files.

I posted this reply to someone else who is trying to help with our workflow.

-----

"No, we haven't been creating subclips. And I checked the project where the sequence originates from for duplicate clips, but did not find any.

Below are screenshots of my workflow and the duplicates (sorry for any redundant info!).

1) via Media Browser navigate to project and import sequence only.

2) Within the new folder containing the sequence and any (supposedly) missing clips from my Master Media Project, I find the duplicate clips.

3) We thought that the spanned clips were the issue: when we import the C300 files initially we would override Premiere's import of the extra clip (how premiere sees spanned clips is combining them into one, so we'd see two clips with identical TRTs and TC, but with different names. i.e. A0003201 & AA0003202), or delete the extra clip from the project altogether. With this new sequence import/duplicate clip issue, we thought Premiere was seeing a spanned clip and bringing in the other half of the clip again.

4) But we're also getting duplicate C300 clips that aren't spanned.

5) The original clip and duplicate are almost identical.

6) EXCEPT for the Video Duration: which as far as I can see is Premiere's way of telling us the In/Out Duration.

Please let me know if I'm missing anything!

Did you run into this?"

[Joe_G!]

Hi Kevin, and whoever else might need this,

We found out what our specific issue is.

In our Master Ingest Project we checked "Write XMP ID to files on import" for all clips, most of which were C300 files.

So all of our duplicates and originals shared the same XMP file.

We went through all the different modifications you could make to the clip, trying to see if there was a main difference beyond Video Duration, and one of us saw that the Audio Channels were different between the two clips.

When importing C300 files we set a rule to change audio channels from the native Stereo / 1 Channel format to Mono / 2 Channel format. A couple of these clips were missed (probably on days where the pressure was on) and that is what made Premiere think there were two different clips.

The master Media Project was not modified, but the editor modified his clips, and this was not matched on our end in the Master Media project. So when we imported a sequence from the editor we got duplicate clips.

We think that when a clip is interpreted at the Project level, then Premiere sees a Project level clip and Timeline level clip as two different clips when importing a sequence.

Hopefully this helps anyone who might be having issues. Kevin, your posts did help out team out a lot when it came to figuring this out.

Thanks for the assistance!

[R Neil Haugen]

Great sleuthing there!

Neil

[Kevin-Monahan]

Hi Joe,

Have you ever considered moving to Team Projects? I'm thinking you'd not be experiencing a lot of this craziness if you did so.

Thanks,

Kevin

[Joe_G!]

Hi Kevin,

Sorry I missed your reply from years ago. I'm only posting this now because I recently started using Productions (I assume what Teams has become) and it's great. Thanks for the suggestion!

- Joe

[andrea_stewart]

I hope this is fixed in the next iteration coming out this fall (2018)

[rachelcenter]

" You need to enable writing XMP ID on import"  where can i find this in premiere?

[R Neil Haugen]

Preferences/Media dialog box.

Neil

[Joe_G!]

Hi,

I'm working on a similar project: multiple computers/projects using the same OS & PrPro version 2015.4.

We get duplicate clips when importing sequences from within projects through Media Browser (or if we import a sequence that was exported as a project). We don't get duplicates for all clips, and the main difference between the duplicates and the originals is the Video Duration (same as In/Out Duration that you would see in AVID).

We think when Premiere is importing these sequences it is referencing the In/Out points (Video Duration) to determine if the clip is new to the project or old. Did you find the same Video Duration difference?

What were you referencing when you said "I think we've figured out how to prevent dupes going forward."?

[R Neil Haugen]

The whole sentence is:

"After reading through this guide: Reuse clips and work across multiple projects | Adobe Premiere Pro CC tutorials I think we've figured out how to prevent dupes going forward."

There are two video tutorials on that link by Max Jago, a very good teacher of video-post. Have you checked those out?

Neil

[Joe_G!]

I know & I did: these videos did not answer my problem. He does have a great tutorial for bringing in new clips, and help if you WANT duplicate clips, but these videos do not tell you what to do in case you are getting duplicate clips you don't want.

I do not have "allow duplicate clips" checked.

[anthonyb87555220]

Not sure if anyone else figured out this method, but I will post anyway (just in case).

I have a similar situation.

Scenario:

Multiple editors working from the same media on different machines.  When editor 1 finishes a sequence(s), they email the project file to editor 2.  Then, in many cases, a tweak is made and editor 1 resends their project file.  Importing the sequence(s) often times leads to a forced import of duplicate master clips despite leaving "Allow importing duplicate media" unchecked.  As you know, this could be a logistical nightmare having multiples of the exact same file.

After researching, I found and article that describes the way Premiere works when importing sequences.  When leaving "Allow importing duplicate media" unchecked, Premiere looks at the available media and determines whether a clip in the imported sequence is the same by checking all file data including original file location.  If it sees any discrepancies, Premiere imports a duplicate clip thinking it's original. (or something to that effect)

On a hunch, I did the following and successfully imported a project without having the duplicate clips.  Here are the steps.

1. close your "working" project

2. open downloaded project master file.

3. relink all clips.

4. save project.

5. return to your "working" project.

6. import downloaded project (or selected sequences).

7. be sure to leave "Allow importing duplicate media" UNCHECKED.

When relinking each clip to the same file as your "working" project, this eliminates any discrepancy issues.  So, when importing new sequences from other projects into your "working" project, Premiere recognizes that the clips in the imported sequences as the same, it does not import the duplicates.

Doing the relinking process can take be quick or slow depending on how organized your media is, but its better than having duplicates.

[Kevin-Monahan]

Hi Anthony,

For collaboration and moving into the future, we are now recommending moving your staff to Team Projects. Is that a possibility?

Thanks,
Kevin

[torchia1667]

The fact that Premiere "Pro" does not allow to at the very least manually indicate that two clip references are in fact one and the same is to me yet another highlight of how badly designed software it is. Also, relinking any larger volumes of assets beyond few dozens/hundreds at a time brings random crashes.

The number of bugs and frustration this program keeps offering through the years is just stunning.

[KEVINCAMPHUIS]

Legend! This worked for me after years of struggling with this issue. 

[alexl45094283]

I FOUND A SOLUTION! It worked for me at least. It takes a bit of time but for projects that are huge like mine, I think it's worth to spend some hours "debugging".

I used the last version of premiere cc 2018.

Here it is how i did it:

1) Make sure that in the original project with dup master clips you have the “Write XMP ID to files on import” activated.

2) Create a new project (I didn’t close the original one given that 2018allows you to have several projects open).

3) Make sure the “Write XMP ID to files on import” option is activated in the new project.

4) Open Media Browser.

5) Make sure that in the media browser settings you have the option “Allow Duplicate Media During Project Import” Off, unchecked, i mean, NOT selected!

6) Browse for the original project in your hard drive using Media Browser.

7) Drag the sequence from the original project to the root of the new empty (use media browser. If you drag it from the original project opened, didn't work for me). It will create the folder structure where the master clips are.

8) Drag the other sequence (the one your other editor prepared and used to point to the dup master clips) to the root of the new project. Voilà! Problem solved. No duplicate master clips.

9) Repeat the operation. You can drag several sequences at a time, but if you drag a lot, premiere wouldn't import anything.

You’ll notice that some times creates the same folder structure, and it seems that is duplicating the clips, but if you go inside the bins, you’ll see that it’s only duplicating the folder structure and inside the bins there are the clips that were not included when you dragged the first sequence.

After that it’s a matter of just organizing the project a little bit.

I had 90 days of shooting, then synced sequences with plural eyes so when i imported them it duplicated the clips. Then also worked with a second editor making cuts of some of the sequences. When i imported those, it duplicated the footage again. So in some cases I had 3 master clips and opening the project and loading clips would take me forever. It took me a couple of hours to “re-do” the project, importing from media browser and organizing it again. But in my case was worth it.

Hope this helped.  

[Kevin-Monahan]

Hi AlexL,

That is a stupendous workflow win that you have discovered via the backdoor of new project sharing feature. Congratulations. I will report this to the team. I hope this will work for others that have the same problem.

Regards,
Kevin

[alexl45094283]

Thank you,

Yes, please, tell the team. I'm pretty sure they can program it and come up with an option in the software like "consolidate duplicate master clips" or something like this, and automatize the whole process i did manually... the result would be a new project where there are no duplicate clips automatically and faster than doing one by one.

Thank you.

[gregorygheorghiu]

If the engineering team is able to parlay this into a "consolidate duplicate master clips" feature, that would truly be cause for celebration. Please publicize widely if and when this becomes available!

[JesseSchluntz]

Can I get a copy of that "shared projects" workflow you wrote up? I'm about to be on a massive documentary project with an assistant editor who will be working remotely and we need to ensure no duplicate media when we share projects. We will each have an Identical RAID. Thanks.

[Trevor_Asq]

One big no-no, in my experience, is amending master clips (e.g. changing audio gain, audio channel assignments, aux timecode etc)

[R Neil Haugen]

Wow ... great thinking there, and swift work of a buggaboo of a problem.

Thanks for posting!