Storage location of large source files for editing on RAID 0 Drive....

[Girocast]

Hi Everyone. I have almost 1 Terabyte of source footage of an entertainer. It's footage of many of his one hour shows, his music videos, his interviews etc....Some of it is high quality HD,  and some of it is low quality SD from VHS tapes.

I plan to pull various segments from all of that footage to create a 15-20 minute video Demo showing his varied work.

The source footage is all now filed on two internal drives on my system (7200 rpm SATA). Also, I just built this system, so it's a very fast, 64 bit Windows 10 machine with 64 Gigs of RAM and very fast video card.

My editing drive is a USB 3, G Raid 4 Terabyte set up as RAID O

My Media Cache Drive is a 1 terabyte USB 3

My Export/Previews Drive is a 1 Terabyte USB 3

Here's my question.

Should I copy ALL of the source footage from the two internal drives over to the RAID 0 drive and start editing? Or, can I leave the source footage where it is and edit the project from the RAID 0? My gut says to copy all the footage over, because then I'd be utilizing the editing speed of the RAID O, but I'm not sure. I just wanted to confirm that copying it all over to the RAID 0 is the correct way to utilize the RAID drive as otherwise I guess my PPro project file will just be referencing the video on the non-RAID system drives, right?

My other idea was to go to each separate source video file on the non-RAID drives,  pull the highlight segment that I want from each video file, and then export that segment to a H.264 file and put those H.264 files on the RAID 0 and edit and assemble the Demo like that. I'm worried though that this would degrade the video somewhat as it's then a copy, so I'm thinking having ALL the source files on the RAID 0 would be better.

Any suggestions? Thanks.

[Moderator note: moved to best forum]

[RoninEdits]

Should I copy ALL of the source footage from the two internal drives over to the RAID 0 drive and start editing? Or, can I leave the source footage where it is and edit the project from the RAID 0?

that sounds like you have a raid 0 internally as well? if yes, then it should be slightly faster. if not, then the external raid might be slightly faster, but it would depend on the drives as well as the raid controller inside the G-raid. there are several free storage benchmark utilities, like cystaldiskmark, which can show you how well the raid setup and other drives you have are performing. since you are working with HD and some SD, it might not require the raid speeds to playback ok. raid could help if you are doing some multi-cam or multi-clip format.

My other idea was to go to each separate source video file on the non-RAID drives,  pull the highlight segment that I want from each video file, and then export that segment to a H.264 file and put those H.264 files on the RAID 0 and edit and assemble the Demo like that.

you are right to worry about h264 degrading the source footage. using dnxhd or cineform instead of h264 would help retain quality if you were to do that. i would only go down that path if the source media was too large to fit on the raid or other destination, but it sounds like that's not a problem for you. it may be easier and faster just to create subclips in the project to keep track of favorite segments, you could also have a separate timeline/sequence of just those favorite segments to then copy sections into a different editing timeline.

[Girocast]

Wow, thank you for that info, RoninEdits. No, my RAID 0 is the Graid on a USB 3. Port. Someone else mentioned I may have too much on the USB ports to utilize the speed I should have. Here I am thinking my RAID 0 is doing the job, but in the back of my mind it did occur to my that too much stuff was on the USB 3 ports. I never looked into what ports are part of what Lanes, but I must do that.

I'm just very confused as I don't want to start this enormous project the WRONG way, even though I'll probably be able to complete it somehow by trial and error.

Again, I need to pull 5 to 20 seconds clips from completed PPro projects that are all about 1 hour in length, and those completed projects are all sitting on my internal drives that are my Archives (of course I've got several external drive backups of those off site).

If I only use the External USB 3 RAID 0 to assemble the project, and leave the Archived projects where they are and just have the Project refer to those clips, I guess that's ok, but I will then be forced to LEAVE them in that exact location or else I'll be relinking in the future should I want to work on them again, which I hate to do as you never know what can go wrong, in my experience anyway.

After what you said about DNx, now I'm thinking maybe I should open those Archived projects one by one, and then export the clips I want in the DNx format so I don't lose quality, and then dump all those DNx files on the RAID 0 and assemble the project there, like that. For some reason, I'm feeling that even though this will be way more work (because I already have H.264 exports of all the Archived Projects already), this would be better. What do you think? I've been using Prem for many years and never noticed the DNx export option in Media Encoder. I was using uncompressed avi to save the best possible versions for Archiving but the file sizes are enormous.

Any ideas from you would be appreciated. Thanks again.

[RoninEdits]

using a disk speed benchmark utility on the usb raid should show how its performing. if the usb ports are an issue, it should be reflected in that benchmark.

as far as location of the files for returning to the project at a later date goes: i wouldn't worry about that part, re-linking media is normal for returning to archived/old projects. its more likely you would have problems with a future version of premiere opening the project or having to install some old version of premiere to read the old project file. several new versions of premiere have used different formats for project files, which have lead to compatibility issues as well as sometimes corrupting projects. the best part is they don't even tell us when they change things and put our old projects at risk. fun times.

h264 vs dnxhd or cineform: if the h264 files are good quality, high bitrate, then they might be fine to use and might have only a little degradation from their sources. if they are lower bitrate or quality, or you want to avoid h264 degradation, it might be good to redo the export as you mention and just export out the parts of the timeline you want/need as dnxhd or cineform. also, dnxhd/cineform is much easier on a computer to edit vs h264. so if you want better editing and seeking performance it might be worth the trouble now to redo the exports or transcode clips from the h264 files.

[Girocast]

Sorry for the delay responding, been sick and then busy trying to make a living.

I'm dreading having to do all the exporting again as I thought the H.264 would be fine to use as I use those for Blu Ray disks, so I figured they are high quality. I mean, they do look good, but I sort of want to follow your suggestions.

Again, my original project files that I would have to export from are all scattered across internal HD's in my system that I use for archiving, so I'm thinking I'd have to move them all back to my RAID 0 for better performance to pull the clips.

Also, I did look at the export setting in Media Encoder 2017 (latest version) and I do see a export choice (I'm on Windows 10) for Dnxhd, but if I remember correctly, there were many presets and I don't know which one to choose.

I'd want the highest quality and file size is not an issue, so if you could please guide me on the Dnxhd and preset settings, I'd appreciate it.

Thank you very much.

[RoninEdits]

if the h264 files have somewhat high bitrate and look decent, it might be fine using those. then using dxhd would mostly be for easier editing as dnxhd would place less burden on the cpu, but more on the hdd(s). dnxhd 175/185/220 bitrate is usually a good balance between quality and file size/hdd performance impact. going for higher dnxhd bitrate or dnxhR hq/hqx/444 will offer higher quality, but might not be very noticeable and could have performance impact on the hdd's if too large of bitrate. you may want to export/convert a few clips in a few different preset options and test them for yourself in a mock project, before committing to a preset and finding out performance or quality issues later.