Productions is always creating 'Recovered Clips' bin with duplicate media

[OliverFelixDear]

Hi all, I'm an assistant editor currently working on a feature doco and we have chosen to use Premiere Productions. Within the production, there are many projects but I'll simplify it by saying there is a Media project that contains the source clips, and a Sequences project that should contain sequences of all the media on a timeline. The Media project also contains the original copy of these sequences - and I am copy pasting this sequence over to the Sequences Project. When doing this, I can go into one of these sequences in the Sequences project, righ click reveal file in project - and it will take me back to the Media project as intended. However, the moment I close the Sequences Project and re-open it, a new bin named 'Recovered Clips' is filled with random clips from the sequences, and the copied sequences no longer point to the clips in the original Media Project, but to this terrible new Recovered Clips bin. Whats going on and how can I fix this?

Some other information:

-We're using Merged Clips. I know, I know, its advised that people dont use Merged Clips anymore and should use Multi-Cam clips, but we've gone down this path now and its too late to go back with the amount of logging, and assembling thats occured. The clips that appear in the Recovered Clips bin are always Merged Clips as far as I can tell (which is to say, a majority of our media). BUT there are many times that sequences filled with Merged Clips pulled into the Sequences Project never create Recovered Clips. 

-Not all clips in a sequence appear in the Recovered Clips bin, and it seems to be differant everytime. I did several tests of dragging the same sequence with a full day of media from the Media Project to the Sequences Project - then closing and reopening the project to see what clips appeared in the Recovered Clips bin that is created upon reopening the project. Always seems to be differant clips. I also noticed that it seems the longer I leave the Sequences project closed, the more Recovered Clips appear in the bin.

-Our setup is 3 iMac's connected to a shared network. The media and project files are all stored on a raid system storage that we are all connected to. 

-Initially, we were all on individual Project files that I have since collated into one Production. I did experience some problems with Recovered Clips as well during the import process, but it was minimal and I just replaced a few clips within the timeline. So perhaps there is some other issue inherant with the media not associated with us being in a Production.

Any support or suggestions on what may be causing the issue is highly appreciated! I'm not really sure how to proceed from here as we cant continuously be creating Recovered clips that arent associated with the original media.

[R Neil Haugen]

@mattchristensen might be able to help on this ...

Neil

[Remote Index]

Hello OliverFelixDear,

This all sounds annoying.

One question I have to add to the detailed information you have provided: If you allow PPro to create the recovered clips on any give sequence (ie. drag a sequence to "Sequences" project, close and re-open) ... what happens if you then take the newly generated "recovered" clips and drag them out of the "Sequences" project and into the "Media" project. If you do this and save and close/quit and then re-open, does PPro regenerate even more additional recovered clips in the "Sequences" project?

I have not had issues to the extent that you are describing, but when I have had new "recovered clips" generated on sequences, it usually only happens on import of a sequence. In these cases, sometimes I will simply file away the duplicate clips in the appropriate project.

R.

P.S. one side question: when you say "I know, its advised that people dont use Merged Clips anymore and should use Multi-Cam clips" - who advised you this? where did you hear this?

[R Neil Haugen]

Merged clips was designed as a quick & dirty on-sequence process to align video/audio. It has numerous limitations, like can't be used in XML/EDL/AAF workflows, you can't use the 'top & tail" Q and W shortcuts ... and others.

The "normal" means of joining separate audio to video is the Multicam process. Which ... I think should have been called something else if that was the general purpose, but ah well.

It's gone into in some good and useful detail in this most excellent pdf on general use of Premiere.

Adobe Long-form and Episodic Best Practices Guide 

Jarle’s blog expansion of the pdf Multicam section: Premiere Pro Multicam

Neil

[Remote Index]

Hello R Neil Haugen,

A better resource on the limitations of Merged Clips is the online user guide - it's more comprehensive than the Best Practices Guide you linked to (though it's incomplete in that it does not list a prohibition on using proxies, and the specific problem with AAF, both of which are listed in the Best Practices Guide).

Nonetheless I am interested to hear the source of information for the original poster.

R.

[R Neil Haugen]

It's been thoroughly discussed on this forum, by Karl Soule in his live presentations and YouTube vids, by David Helmly in his live presentations and advice to the "Hollywood" type & broadcast users.

Among other places.

Edit: But for all that it sure isn't obvious or what anyone would expect by the names.

Neil

[OliverFelixDear]

Hey guys, yeah it partly was the Best Practices Guide from Adobe that I was sourcing that info from. However I was also in contact with an Adobe representative for an internship earlier this year who also advised not to use Merged Clips and that Multi Cam is what they want to be promoting as the proper method for linking audio/video. They acknowledged the naming confusion too, as 'Multi-Cam' doesnt sound like the right terminology. I only went Merged Clips on this current project because the editors had expressed their problems with Multi-Cam in the past, but Im not entirely sure what their grievances were - something to do with deliverables.

I am still testing some of the methods suggested above for solving my current problem.

Oli

[Remote Index]

OliverFelix Dear,

Thanks for the info.

I was going to suggest investigating the "Edit > Reassociate Source Clips" and the "Edit > Consolidate Duplicates" commands as possible assistance, but they would not address the core issue (the generation of the duplicates) and actually I'm not even sure they are relevant to your issues, nor how they interact with Merged Clips. Perhaps someone with more experience using these commands will offer some insight.

I'd be interested to know if you come to any resolution or new information on this in future.

R.

[OliverFelixDear]

Hey there! Thats great to know about the 'Reassociate Source Clips', that may definitely help with solving some of the existing problems duplicates. Consolidate Duplicates has never worked in my instances. 

I tried as you suggested above to test what happens when placing the Recovered Clips from the Sequences Project back to the Media project. It does not recreate more recovered clips when I close and reopen the Sequences project, so thats somewhat good if I was happy to keep the dupes. BUT it seems to open up a whole new can of worms, whereby many of the sequences in the original Media Project that contained instances of those clips - now link to the Recovered Clips. Trying to backtrack those steps to get them relinking to the original merged clips has been a whole other journey for me this morning, so I'm going to avoid moving Recovered Clips around now.

One thing I think may be causing this problem is that perhaps renaming the merged clips is causing the dupes. For example, lets say I create a sequence in the media project with some Merged Clips rushes. I then rename one or more of the Merged Clips after the sequence has been created because I might have noticed an error in the log name. I then drag/copy that sequence to the Sequences project and initially everything is fine. Until I close the Sequences project of course and viola, Recovered Clips - potentially the ones the I have renamed. I need to do some more tests to see if thats the core issue - because I spotted some slight variations in the naming between the Recovered Clip and the original. 

What I've been doing to mitigate Recovered Clips being created, is to recreate the sequences from the original Merged Clips again - and only then bring them to the Sequences project. I cant recall everything Ive done since creating the original sequences, so its hard to know if its renaming the clips, or something else that causes the problem. Potentially something I had done before the project was a production? For the most part - I stopped getting recovered clips after remaking the sequences. This obviously works for simple things like a Rushes sequence etc, but wouldnt be a long terms solution when we start moving edits/assembles between projects.

The Recovered Clips issue still also happened to a few shoot day sequences no matter how many times I recreated them. In those instances, I found that those shoot days actually already had some existing duplicated in my Media Project, way back from when they were still in standalone projects. So perhaps if existing duplicates are brought into a production, that can cause problems when moving sequences containing them between projects.

Another observation Ive made is that once I do get a successfully get a sequence into the Sequence Project, it'll never produce Recovered Clips from there on. I'm hoping that if I can successfully pull everything across, everything contained in those sequences (which includes all media for the entire production) will then be safe to copy/drag to another project forevermore.

None of my theories fully explain why random, differant clips are selected every time I bring a sequence in to the Sequences project though. So its still hard to pin down exactly what the correlation is.

I'll continue to post my experiences with this project if I find out more.

Oli

[mattchristensen · Accepted solution]

@OliverFelixDear Thank you for the detailed description of your setup and issue. More than likely this is happening because of how the separate projects were combined into a Production. If you're going to take an existing set of projects into a production, it's important that you first combine everything into a single project. Get everything organized, duplicates removed, etc, in that single project, and then break it apart into a production.

If instead you create a production and add separate projects, you could end up in a state like you describe. The Recovered Clips bin is Premiere Pro's way of catching clips that have lost track of which project they came from. So how to proceed from here depends on which of the following steps makes the most sense for your project. I would suggest doing one of these:

1. Gather everything into a single project in the production. All sequences, source clips, etc, all in one project (you can keep the bin organizations). Then run Edit > Consolidate Duplicates to catch any duplicates. Put the other (now empty) projects in the trash. Save the project, then break apart the production again, making new projects and moving clips and sequences to their appropriate projects. Keep all the projects open as you do this, saving often. When it's all done, you shouldn't see Recovered clips appearing anymore.

2. As Recovered clips appear, move them to their appropriate media projects. If, when you do that, you find what look like duplicates in the media project (the "original" and "recovered"), try running Edit > Consolidate Duplicates. It may or may not work, depending on many factors I won't go into. If they don't consolidate, you'll just have to live with having two copies of the clip in your source project. This is because they came from separate original projects, and some part of their metadata is different (markers, other metadata, etc), so Premiere Pro doesn't see them as duplicates which can be consolidated.

I hope that helps!

Matt

[R Neil Haugen]

Thanks Matt! That actually answered some of my wondering also.

Neil

[OliverFelixDear]

Thanks Matt! I havnt given this a crack yet - fully making a new project with everything inside - but it does sound like the logical next step with everything I've experienced and tested so far. 

For context on how the project was created for anyone reading this in future - I did start by mostly having all media / sequences in one project and then split it from there. The problem I believe started though when another editor working independantly handed over their project - and they had been facing problems with duplicates previously. It only had one shoot day of media so we thought it would be as simple as bringing those sequences over to the production and dealing with those shoot day duplicates that might occur. However, I've still been getting duplicate clips from the shoot days that only ever existed in the 'master project' when copying sequences outside of it. I wonder whether involving these other editor's projects with their duplicates has somehow 'infected' the whole project in some way.

One other thing I have noted - is that 'copying' a sequence to another project without source media more than often produced Recovered Clips. Whereas dragging the sequence over generally did not. Copying it back to the project housing the media after the fact was ok. 

Thanks everyone that has given suggestions - I'll post an update after recreating the production as Matt suggested.

Oli

[Mariwenn27980842691n]

Hi ! I apologize in davance for my english, I'm a belgian asssitant editor and my english is a bit rusty.I'm not sure it will help, but I discovered another way to get rid of these Recovered clips (I had exactly the same issue as you)

If you clic on Edit -> Consolidate Duplicates, then create a new project inside your Production, called "Recovered clips, then drag and drop the recovered clips bin in this project,  and from there delete this bin, Premiere then stops creating Recovered clips when you reopen the project.

And if you do a "Reveal in project", it will just ask you if you want it to search in the others Project, and will point to the original Merge Clips.

You can then do Edit -> Reassociate Source Clips and select the right project.

I still have to do some workflow test to make sure it doesn't affect the relink to the full res medias at the end of the editing, but for now, it personnaly saved my day and I hope it will help you.

[jacekk]

I have the same issue but none of these solutions and causes apply.

Around 30-40 Photo and Music files are 'Recovered' every time I open the Production main edit project, which makes me suspect a software flaw more than anything. 

I have a Production with 6 large Projects for a 6 hour/episode documentary series.

I followed Production Best Practices for setting up the Production from scratch.

Hundreds of photographs in one project and equally as many music tracks in another - problems - causing 'Recovered Files'.

110 hours of interviews and synced audio files in another - no problem.

Hundreds of B-Roll clips in another - no problem

Yet, Photos and Music are constantly 'recovered'. I have relinked them to the masters at least 10 times, and yet the next time I restart the project, they end up in the 'Recovered Files' folder in the main project. Nearly always the same files with some new ones added.

There are no duplicates on the drives or in the project. From the above, I tried 'Consolidate Duplicates' nothing happens.

It's not causing problems other than files now being duplicated in the recovery folder - I am just trying to prevent further issues down the line.

Any progress and solutions since the last posting?

[R Neil Haugen · Accepted solution]

And to check ... the files whether stills or video are on the same drive and folder tree? Probably different subfolders?

[jacekk]

Yes. Different Drives. Different subfolders.

This a massive project with media on 4 x 4TB individual drives.

I did however find several old 'recovery' projects in the Production trash folder. Probably left over from previous crashes.

I'll check on start-up tomorrow if the usual suspects 'Recover' themselves again.

That may just be the source and cause of the problem. Hopefully.

[jacekk]

But if I understand your question correctly, the 'recovered' files and the original files both point back to the same single file on a hard drive. The main edit project is creating a duplicate on start-up. 

[jacekk]

I just re-opened the project - the same 37 files all 'recovered' themselves.

Before opening, I trashed all the trash files, then opened the Production, then the Music and Photos projects, and then the Master Edit project. Thinking it may want the 'feeder' Projects opened first.

But it made absolutley no differnce. Always the same files.duplaicte themselves in the Main Edit project on opening. I give up.

[R Neil Haugen]

Just thought I'd ask another potential issue ... what is the full folder-name/file-tree for these files? Is there any character used that is different than the others?

[jacekk]

Yes Neil, the folder structure is very basic two levels deep, but the Photo file names have long, many multiple characters. They come from multiple sources and because I have over 10,000 photographs (98GB of jpgs), I need the reference to the source in their names. Only about 200 photos are in the project though, so far - I select in the finder and only import the ones I use in the actual timeline t avoid ssues with so many files.

However, the music files which 'recover' themselves have a very simple folder/file name and structure.

Thank you for your interest. As I said, at the moment this is only causing a problem with my Project Tidiness OCD, but as this project fills out this may cause issues in the future.

[R Neil Haugen]

Any non-letter/numeral character can be problematic. As can names, if the total characters between folder tree and filename is past a certain number.

[jacekk]

Certainly, but why these specific file names out of the hundreds with similar names?

And not all of the 'recovered' files have long file names. None of the music ones do.

Seems arbitrary.

Any idea what the non problematic number of chartcters limit is?

[R Neil Haugen]

A couple others around here would know that.

Anything other than a letter or number is unwise of course.

[Remote Index]

"Any idea what the non problematic number of chartcters limit is?"

"A couple others around here would know that."

Clear documentation on this would help users.

R.

@Kevin-Monahan(documentation issues)

[Jane28708435jkl5]

I am having a similar problem but in addition to Recovered clips being created, it brings in the source clips in their original folder structure created in my "media" project.  So in a new project within my production, I bring in a sequence (or add a select to a new sequence in the project) I get both original media brought in and Recovered clips. I can try the solve above, create one master project, but I don't know if this will solve my issue as it's different than just recovered clips. I have a massive production with over 25 projects so this task will take a long time. Trying to see if I have a different problem. At some point, I had to revert to a backed up version of my production and had to turn it into a production again (there was a prompt for making it into a production). I didn't notice the problem right away but including this info in case that's the culprit.