Conforming and Indexing Errors, Media Pending, Audio won't play in timeline

[thesteng]

I'm working on a desktop PC which is running Windows 7 Professional 64-bit and Adobe Premiere Pro (version CS5.5). It's currently utilizing a second gen. 3.4Ghz i7 2600 processor, 16GB of 1600Mhz RAM, 64GB solid-state drive and a ASUS P8Z68-V Intel Z68 Motherboard with onboard audio (Realtek ALC892 chipset) and onboard video. My problem is this:

The conforming and indexing of all of my imported media never seems to finish regardless of how many times I reopen the project file and wait for it. On the lower right-hand portion of the screen, next to the conforming/indexing progress bar, is a little red "X". When clicked, it pops up with a list of errors that read: "An unexpected error occurred while performing a conform action on the following file...". As a result, my audio channels have no waveform and during playback there are no audible tones or levels. On some video clips there's just text that reads "Media Pending". This only appears to happen with project files that I saved on external hard drives, and I suspect it has something to do with the Media Cache Files folder and how Premiere Pro locates these conform/index files. I've also encountered this problem in CS3 and CS4.

I have a few questions:

1) How do I avoid error messages in regards to indexing and conforming

2) How do you know when indexing/conforming has completed itself? (there doesn't seem to be a progress log or a list of commands/executions)

3) Indexing and conforming appears to be an automatic process, but is there a way to do it manually?

4) What's the best way to setup your media cache files when you click EDIT > PREFERENCES > MEDIA?

5) If I have approximately 1 hour of footage, what's an average wait time for conforming/indexing? What about 5 hours of footage? 10?

6) Adobe recommends not editing until the conforming and indexing has completed itself-- how important is this?

7) Sometimes it appears as though the conforming and indexing has finished, but then I still have problems with playback. Do I have to reopen the project for it to continue with the conforming/indexing progress? I've already determined that the video file I'm working with is intact and free of any corruption.

I'm fine with having to wait for a project to conform and index, but it never seems to complete itself! Any help regarding this matter would be greatly appreciated.

[Harm_Millaard]

The first problem when using externals is that you have to explicitly define a fixed drive letter to that external in Disk Manager. That way projects can always find the media. Media pending is often caused by slow disks but is usually of short duration, a couple of seconds, unless you use USB externals, then the time may be much longer.

Indexing, conforming and peak file creation is an automatic process and must be finished before you start editing. You will see the progress in the lower right hand corner of the screen.

Indexing is required to speed up GOP editing, conforming is required to edit the audio and peak files are required to display waveforms. If that process is stopped before it is finished, you can run into video and audio problems. As to where to setup your media cache and media cache database, have a look at the Generic Guideline for Disk Setup in the FAQ section under the Overview tab in the Hardware forum.

To solve your problems, I suggest you completely clean your media database, change the location to a different internal disk than your C drive, load your project and let the indexing, conforming and peak file creation finish. The time required for that process depends on how fast the system is, the codec of the source material and the length of the material. Therefore it is nearly impossible to say how long it takes, but for comparison, a 6.5 hour AVCHD with 5.1 audio loads and indexes in a matter of minutes on my system.

[Travis__]

Non-effective. The first thing any of these gentleman are told to do with conforming issues is to clear the Media Cache, etc.

And that often creates more problems than it solves when you have a pending project(s) that you re-open & allow to re-conform. This whole process is so buggy & unpredictable & I'm a hair's breath from completely giving up on Premiere, the development of the conforming process is so poor it had no business even being released in this state. And now they're concentrating almost exclusively on CS6 development, using all the complaints, screen clips & feedback we the users have provided, only not to introduce a solution for CS5.5. Instead, we have to wait for the possibility it may be resolved in a subsequent version.

And exactly what might compel a person to buying the new release when you can't solve something so problematic in the version we're already using??!

Adobe?????

I've written here only to assert that this man is NOT full of it, he's not doing anything wrong & the problem is entirely PREMIERE. Not his own ways & means or how he's using the program.

Those who are not encountering these problems must be doing drudge-work with the same kinds of material & in short productions to not be running into audio conforming issues. Because I encounter them with EVERY SINGLE PROJECT.

Sick, sick SICK of it.

[shooternz]

FWIW. I  (and I assume most others) dont see the issue ... and I dont do "drudge work".

I also edit files more designed for editing than mpeg-2 and mp4 and I rarely use subclips.

Have you tried different source files?

Conforming when I first load source files takes seconds and does not re conform next time the project is opened.

The O.P in this thread has had some areas identified by Harm that seem logical.

Be interesting to see how he fares.

[Travis__]

Indeed I have tried using different source files, I've tried everything imagineable before uttering a single complaint or seeking advice. I use subclips a great deal because I do not think I should have to render-out individual subclips to be used in a sequence when subclips are intended for this express purpose. An example of things that I've tried is reprocessing the problematic clip through AVC Video Converter, inputting & outputting the same frame size, etc. To get through any potential blips in the originals, even when I'm certain they're fine.

I also am not using 'Write XMP ID to files on import' because that has screwed up a number of long-running source files unto itself.

IMO now, after using Premiere/After Effects extensively for several months, that Premiere (in CS5.5) in particular was broken from inception, insofar as management of clips & conforming audio is concerned. These problems may have been made more severe with the introduction of the 5.5.2 update.

Such a pity that such a great conceptual program can be a total P.O.S. in this one critical areas.

A greater pity is how much time I've wasted with it thinking I could somehow work around these problems.

Forget CS6, Adobe!!! We need CS5.5 FIXED FIRST!! This is non-sense. I'm certainly looking into the BIG jump to Avid but I did not think a package of Adobe's caliber in this price range would be so mucked up. Why am I spending so much more time persuading Premiere into behaving that I am tweaking my projects & rendering? So fed up. No words...

[Wil Renczes]

Harm filled in pretty much all the salient details, but I'll do another pass here.

1) How do I avoid error messages in regards to indexing and conforming

Two parts here.  One, conforming only happens for certain media files, ie the ones where performance is critical and we can't depend on extracting the audio fast enough for realtime playback.  That's basically anything in an .mpeg wrapper, or AVCHD material.  So if you edit XDCAM HD/EX or P2, or RED, or even AVIs or QT, those formats don't require audio conforming.

If you're stuck editing AVCHD or MPEG2, then it needs to conform.  But, that being said, you shouldn't be getting errors in the first place. I think it's related to your external drives.  More below...

2) How do you know when indexing/conforming has completed itself? (there doesn't seem to be a progress log or a list of commands/executions)

Nope, you have a progress status bar indicating which file it's working on.  If there's an error, it shows up in the events panel.

3) Indexing and conforming appears to be an automatic process, but is there a way to do it manually?

No.

4) What's the best way to setup your media cache files when you click EDIT > PREFERENCES > MEDIA?

While some people like having the check box for having the conform files beside the media, I hate it.  Yes, it means that if you move the project to a different system & reopen, it means that you potentially can avoid recreating CFA files, but I find the drive littering not worth it.  I much prefer having setting the Media prefs to point to a specific media drive.  Usually a raid, if available.  Definitely not an external drive that you disconnect & walk away with.  If you don't have a permanent raid on your system, then preferably a dedicated internal drive for media (think along the lines as your Photoshop 'scratch disk').  Failing that, leave it on your C: drive, although with a 64 Gig SSD, you probably don't have much room for transient temporaries.

5) If I have approximately 1 hour of footage, what's an average wait time for conforming/indexing? What about 5 hours of footage? 10?

Like Harm said.  Totally dependant on the media container & the speed of your drive i/o.  The conforming is iterating through the entire file & pulling audio data, so it's not CPU intensive, it's all i/o.

6) Adobe recommends not editing until the conforming and indexing has completed itself-- how important is this?

If you're trying to play/scrub while conforming, it's going to be pokey.  Esp. if you're trying to access the file that's actively being conformd.  As I just said, we're hitting the files for all the audio.  The i/o is being saturated already, so unless you have a stellar raid, you don't have much headroom.

7) Sometimes it appears as though the conforming and indexing has finished, but then I still have problems with playback. Do I have to reopen the project for it to continue with the conforming/indexing progress? I've already determined that the video file I'm working with is intact and free of any corruption.

You should be good to go.  Sounds like there's something else at play here.

Okay, back to what I think is wrong:  you don't mention what kind of external drives you're using.  You're making a bad assumption that blowing away conformed files & doing a reconform is buggy - I doubt it, as that's the same process that happened when you initially brought in the files.  I've blown away my media cache folder multiple times and have never seen failures on reconform.  So it's got to be one of two things:  either a read error from the source when attempting to pull the audio, or a write error to the destination.  Now I don't know where you currently are pointing the media cache directory, or what your source drive is, so I can only speculate.

My suggestion is to do some elimination.   Copy one of the files that failed on you to your C drive, & target your media cache directory also to C:.  Pick a new project, import your copied file, confirm that it conforms correctly & behaves.   Then, try to use the same clip from your external drive, keeping the media cache to C:.  If that's still good, then try targeting another (local/internal) drive as your media cache target; close/restart, then import the clip from C:, and then import the clip from your external drive.  This troubleshooting should give us something.

PS, if you're trying to edit from external USB drives, good luck.  I find it a major PITA that I avoid as much as possible.  Firewire isn't much better.  I know some people do it successfully, but I think it's a road fraught with peril.  These devices are generally not designed for heavy duty I/O and a flaky connection or drive is nothing but pain.

Cheers

[Peter_J_Burn]

indexing renders - my personal Hell.

Supermicro X7DA8 board, dual Quad XEON E5420s, 32 gig RAM, two AVID Utra 320 SCSI RAIDs, each with own channel.  nVidia quadroFX 3800.  No horsepower issues.

Just upgraded to CS5.5.2, hoping the endless indexing rendered issue would clear up but, alas, no.  I have an XDCAM HD project with 7500 pieces. It starts the hateful indexing renders dance once I get to 52xx remaining, then that's it.  I can count the individual clips loading.  6 gig RAM in use and two Quad XEONs are running at 30 - 35%.  It now takes over an hour to load the media whereas before the indexing virus outbreak it took a couple of minutes.

Did everything suggested above, to no avail.  I've owned and used Premiere since version 5 but that's it for me.  I don't like the AVID GUI but it doesn't have these bugs so when I finish this job I am done with Adobe Premiere Pro.

We bought a PDW 700 XDCAM HD camcorder but cannot edit the footage.  Too late to redo this in Sony Vegas Pro 10 64 bit so I just hope eventually the media loads so I can make the changes and be done with this nightmare. 

[Wil Renczes]

Out of the supported MXF formats, only 422 variants currently require indexing.  That need will go away in the next release of CS, as we've integrated 422 decode support into our MXF importer (previously it only had native 420 decode support, so anything 422 goes to the fallback path of importing via our Main Concept importer, which requires indexing/conforming).

[Peter_J_Burn]

The media is XDCAM HD 420, not 422.  Unless CS6 is coming out very soon I am going to have problems delivering this project to our already p***d off customer.  It took 90 minutes for the Project to load and once loaded, 16 gigs of RAMand 35% of my CPU are taken up by the indexing renders feature.  If you know of any way to fix this, please contact me directly. [email address removed for privacy reasons] I thought CS4 was nasty but this is even worse!

[Wil Renczes]

If you're getting indexing on 420 MXF files, something is definitely wrong.  This means that the XDCAM importer is failing to handle the file & the fallback Main Concept importer is attempting to do the work. (Just to elaborate on why it's slower through that path:  the fallback importer is an MPEG importer, so it doesn't know about any of the smarts in an MXF file, it tries to pretend its an MPEG file with a bunch of padding, so it does indexing to know where all the frames are located throughout the file.  MXF containers have an index table, so if you know how to handle the file correctly, indexing is completely unnecessary...)

First thing to check, is the importer plugin still loading correctly?  Look for the file 'Plugin Loading.log' in this folder:

C:\Users\<username>\AppData\Roaming\Adobe\Premiere Pro\5.5\

That log file will list all the activity on launch with the plugin loading & should tell us if there's a failure.  Look for ImporterXDCAMHD.prm in that text file, see if it's indicating it failed to load for some reason.

If that seems to be loading correctly, step two would be to take a known XDCAMHD 420 file & attempt to import it into a fresh, new project & see if it seems to load correctly, or whether it starts indexing again. If it still wants to index, I'd probably wipe the Media Cache folders clean & see if it still misbehaves.

[Peter_J_Burn]

Loading C:\Program Files\Adobe\Adobe Premiere Pro CS5.5\Plug-ins\Common\ImporterXDCAMHD.prm

The registry tells us to cache so the plugin will be loaded from the cache.

Loading from the registry...

The plugin was successfully loaded from the registry.

[Wil Renczes]

Okay, so the plugin itself is working, good.  Now the question is which importer is trying to handle your media.  Here's a surefire way to know:

1) right-click on one of your files in the project panel & select 'Properties'.

2) hit Ctrl-F12, this will bring up the debug console window.  If you go to the top right corner & click on the wing menu, pick the option for 'Debug Database view'.  This will bring up a list of various debug flags - look for the one called 'BE.Media.GetProperties.GetImporterTestingInfo' (should be 8 down from the top of the list), and set it to 'true'.  Now close the properties panels & bring it back up, there should be additional information now, and you should have an entry called "ImporterModule path = ..."

If that ImporterModule path is referencing ImporterMpeg.prm & your footage is 420, then we know for sure that the wrong importer is trying to handle your media.

Now, how to fix it:  I don't know what made that importer fail in the first place, but what's probably preventing the correct plugin from reloading the files is a accelerator file in the media cache folder, which is why I was suggesting wiping it.  The softer thing you could do is delete *.ims files within your media cache folder (those are the accelerator files that say whether a certain importer will pass or fail on a given file type, so this is less destructive than blasting the world in here).

Your default location btw for the media cache is here (unless you changed it to point to somewhere else in your prefs):

C:\Users\<username>\AppData\Roaming\Adobe\Common\Media Cache Files\

If you wipe the .ims files (btw please have PPro closed, then relaunch... I'd like to think that's obvious, but it never hurts to be specific...) and the properties window still insists that ImporterMPEG is handling your XDCAM files, then either a) the file isn't somehow a standard 420 XDCAM file, or b) the file is damaged in some way, or c) ... I don't know, there's something else in play here & my crystal ball is too cloudy to make out what. 

[Peter_J_Burn]

The machine with the problem is working in Mercury acceleration instead of going through the AJA Kona LHe.

I think I have figured out what lead s to the indexing renders by going back in time to a version of the Project I was personally working on last year.  Indexing problem not there.  Managed to replicate it by "rendering entire work area" then away it went with the damn indexing rendered process.

In this mode tested on this PC ONLY "render effects in work area" does just that but "render entire work area" initializes the indexing rendered anomoly that makes editing impossible.

I deleted every preview and Media Cache and database folder, and did everything else suggested.  It is okay now so thanks for the suggestions about purging everything.

Regarding the benchmark speeds vis-a-vis performance and such, in my roles of Project Manager and Producer I oversee work done at three locations with AVID Media Composer Adrenaline, Adobe CS5.5/5.0/CS4,  Boris Red 5/4 and Sony Vegas/Sound Forge  9/10/11 64 & 32 on the Quad XEON with the 3800 card, 3 HP xw 8400/8600s c/w nVidia 4800s, a P6NT c/w I7 9?0/GTX 580 and a couple of machines with I5s 32 bit and GTX 4?? or maybe 2?? video cards.  The only performance difference I see is some of the machines render to Windows Media 9 or FLV faster.  When rearranging and trimming clips on the timeline processor speed is not really relevant.

File Path: H:\ASE environmental\disc 5\Clip\C5069.MXF

Type: XDCAM-HD Movie

File Size: 41.7 MB

Image Size: 1440 x 1080

Frame Rate: 29.97

Total Duration: 00;00;09;00

Average Data Rate: 4.6 MB / second

Pixel Aspect Ratio: 1.3333

Compression Type: MPEG

XDCAM HD non-temporal metadata:

Title:

    MAR201002134

*********** Begin Importer Testing Info ***********

ImporterModule path = C:\Program Files\Adobe\Adobe Premiere Pro CS5.5\Plug-ins\Common\ImporterXDCAMHD.prm

DisplayName = ImporterXDCAMHD

Module Priority = 10

FileType =  1297630752 ['MXF ']

Total Stream Groups = 5

StreamGroup Indecies: [A=-1 | V=0] [A=0 | V=-1] [A=1 | V=-1] [A=2 | V=-1] [A=3 | V=-1]

Video Stream Group Iteration :

          Reported codec = 1297106247 ['MPEG'] - Description =

Additional Media Files Reported :

\\?\H:\ASE environmental\disc 5\Clip\C5069.MXF

[Harm_Millaard]

Supermicro X7DA8 board, dual Quad XEON E5420s, 32 gig RAM, two AVID Utra 320 SCSI RAIDs, each with own channel.  nVidia quadroFX 3800.  No horsepower issues.

You might be surprised that a similar system like yours, but with the FX 4800 video card (same CPU's, same memory) performs about 5 times slower than a fast system and ranks at # 420 out of 667 systems in our Benchmark Results. The outdated CPU architecture, the low clock speed and the communication between the two CPU's cause this rather weak performance. So I disagree with your statement 'No horsepower issues.'

If you have a two channel raid controller with two arrays, make sure that your media are on another channel than your media cache and media cache database.

[Peter_J_Burn]

Thanks but this has nothing to do with the indexing problem.  Also, I do have an I7 with a 580GTX, DDR3, 24 gigs RAM and AVID SAS RAID and it is not significantly faster with XDCAM HD.