Generic Guideline for Disk Setup

[Harm_Millaard]

There have been many questions about how to set up your disks.

Where do I put my media cache files, where the page file, and what about my preview files?

All these allocations can be set in PR, so I made this overview to help you find some settings that may be beneficial. It is not a law to do it like this, it is a generic approach that would suit many users, but depending on source material, workflow and backup possibilities, it is not unthinkable you need to deviate from this approach in your individual case.

The reasoning behind this overview is that you want to distibute disk access across as many disks as possible and get the best performance.

Look for yourself:

I hope this helps to remove doubts you may have had about your setup or to find a setup that improves performance.

[PaulieDC]

Hi Billy,

OK, here's answers, as best as I can give them:

Some things I need clarification on:

• I was considering getting another drive (for a fourth in the system) You are recommending a 10K RPM vs another one like the other three I have??? (Currently have three 1TB 7200 RPM SATA drives) YES, a single 10K Raptor for OS and Adobe Suite (and plugins) works great at this point in history. Better-made SSD in the future will probably supplant that). It should have no user files on it at all, so if the drive were to fail, just reinstall OS and Adobe and you're good to go. I just make a ghost image of my Raptor every so often so I have an image with the latest Adobe updates, etc.
• Are you suggesting The OS and Adobe Pr Pro on one drive alone. What about other apps? This is a problem I also have... most guys on here will tell yhou to make a clean editing machine with no other stuff on it. I just can't do that financially or space wise. SO, I simply popped in another drive (I have 7 total in my case, the list is below) and that's where I install all my other apps such as Office and Flight Sim and Visual Studio and SQL Server (I do software development also). This does require me to turn off several background services manually before editing, but that's life. So on my PC, I have the C drive for OS and Adobe, and then my S drive (I picked S for secondary, no other reason) also has a Program Files folder which is an important one as you know with Windows. See, by doing that, you main OS drive (Raptor if you get one) won't get clutter and therefore the heads won't have to hunt around 60 extra gigs of stuff to find what it needs. Seems like overkill, but your single most point of "slowness" are those mechanical drives so you need to do all you can to run then lean. In Windows we turn off compression and indexing also on the drives. BTW, the 150GB or 300GB Raptor is fine, don't spend the bucks on the 600GB.
• When you say RAID 5 in the system are you suggesting the RAID is done on the other three discs with the OS/Pr on the other as a stand alone. RIGHT. You got this nifty RAID5 system, which means three disks with great speed and redundancy (which is why you only see 2TB). You basically need to look at it as just the storage drive for your video footage. Apple intended that drive setup to do everything, but you're going to be uber-cool and yank your OS off of the RAID5 and put it on a totally new drive, then wipe that RAID5 clean and make it storage-only. DISCLAIMER: all regular SATA drives work with your Mac, right? Meaning, a Raptor is usable on a Mac? My last Mac was a Quadra 610 17 years ago and I remember it being SCSI. I'm covering myself here, lol!
• If I have RAID set up can I still designate those drives as separate drives for Pr functions (e.g., one of the drives in the RAID is named Project and that is where the project files are written)? No, the 3 disk RAID will appear as one disk and you lose 1TB in the setup as I mentioned above. How it works: two of the drives work together as one drive, so the two 1TB drives become one big 2TB drive. There's a "stripe" that's written so the two drives stay in sync. You could copy a video file to that drive (let's called the RAID5 set a "drive" from here on in, because that's what you'll be seeing), and that video file could possibly get written partially on one drive and partially on the other. The beauty of making two drives a striped RAID set is S-P-E-E-D. Now, if you stopped there and only had two drives, that would be called RAID0, or commonly called a "stripe set". The big problem is if one of the drives fails, you lose it all. After you replace the bad drive you have to format and rebuild the stripe. SO, that's where the tird drive comes into play. When you add that third drive, it acts as a mirror, so if one of the first two drives fails, that third drive will take over and you don't lose your data (OK, it's a bit more complex than that, but that's the idea). You don;t get to use ANY of that third drive, it's sole function is to mirror your data for protection, which we call redundancy. SO, forget that it's three drives, it's really one big 2TB drive with protection. That's why you can't leverage any of the three drives individually.

I guess I failed to mention that I do have a NVIDIA Quadro 4000 for Mac. SUPER!!! It is not installed yet, because I need to get this whole drive thing settled first before I go and install the card drivers and the CUDA drivers because it sounds like I may need to delete the RAID and start all over again with a different configuration. If you go this route, yes, I would get the OS running on the Raptor and ensure it's all good to go, then I'd format the RAID5 set (or drive!) and start clean with it for your footage.

Also, should mention that I have a RAID card installed in the system as well, so it will be hardware supported RAID not software.SUPER again! On-Board RAID is useless, I'm feeling that pain as we speak.

So, how do I get started from here? Do I need to get the fourth drive, delete the current RAID 5, install the new drive, install the OS on one drive then set up RAID 5 on the remaining drives?

• Buy the Raptor and install your OS and Adobe stuff on there, and make sure it all runs jim-dandy
• Wipe the RAID5 set clean. That's your new footage or "Asset" drive. You don't have to delete the RAID setup or anything, it's already running. Just format so your original OS and all that is gone.
• Buy a good 7200rpm drive for saving your Pr project files and for output files. I like the 1TB WD Caviar Black.
• Buy a good 7200rpm drive for all of your other programs and personal files and utilities and photos, etc. PLUS, create a folder on here for Adobe Scratch Disks. Spreading the Adobe load over 4 separate disks is a great way to go.
• Shoot, edit, and be happy!

I would like to discuss your suggestions of formats for editing (all very good advice) but I will wait until I get this thing off the ground and then tackle that. I think once I get that all going, I will be ready to roll. Unfortunately, all of this is holding up a deadline for a publisher. But as you can see, it makes no sense to do anything until the drives are set properly so that I don't need to restart all over again since RAID set up will require a wipe and start over.

Hope that helps! Let us know!

Paulie

[PaulieDC]

Oh, I did mention I would list my drive setup, here it is, ALL WD drives:

• 300GB Raptor: OS & Adobe Production Premium Suite
• 320GB 7200rpm: second drive for program installation (non-editing), plus Scratch Disks
• Two 1TB RE3s in RAID0 for assets
• 750GB Caviar Black for Video Project files and Output
• 750GB Caviar Black for all Photos (nothing to do with editing, but I'm also a stills guy, so this is my photo vault)
• 1TB Caviar Black: this is for all of my other stuff... installation files, tutorial videos, iTunes Library, you name it. If it's junk, it goes here. Since Windows likes to make the standard Windows user folders (My Docs and My Photos and My Videos and My Music) live on the OS drive as part of your profile, I make sure to keep those empty. I make duplicates on this 1TB Junk drive and store all that stuff on here. Then in Windows 7 I just make Libraries for each and point them to the Junk drive, and viola, my OS drive is lean! Oh, I also put my swap file (the Windows Memory Paging file) on this drive so it's not eating up 18GB on my OS drive.

I also have a couple of LaCie Quadra aluminum externals (one on eSata) for backup.

And, just for interest's sake, I'm running an Intel i7 950, 12GB ram and use an EVGA GTX470 card. To finish it off I have a Pyle stereo amp, two Polk Monitor 30s and a huge JBL Balboa subwoofer for audio. My next goal is a RAID card and a better IPS monitor... using an older Dell 2405 right now.

That's it!

-Paulie

[Guest]

Paulie,

Thank you again so much. Your time with this is a great help to me and I am sure many of us on the forum that are trying to get up and running with the exciting world of Pr CS5.

So, if I understand you correctly, I am going to keep my three 1TB drives set as RAID 5. Wipe them clean. Get the 10K Raptor install it on the system and put my OS and apps on there. ( I need a place to run other apps. I am also running Logic Studio as I do audio recording also). My case only supports 4 SATA drives. I have an external iOmega 2TB with FW 800 that I will use for system backup and storage. Is this right?

Bill

[PaulieDC]

Being you have a 4 drive limit, that's the way to go. Raptor for OS and all of your apps, and the RAID5 drive as your big fast hairy storage vault. Separating your assets from your OS/Apps is an important one!  I would get the Raptor first and get your OS and all of your apps installed before wiping the raid, just to be sure.   Check Newegg.com, they've been having great prices on the Raptors. If I find a promo code, I'll email it tonight.   Have fun!

[Guest]

Just to be clear, when you say wipe the RAID, you are saying getting the OS and Apps off of there but keep the RAID set up correct?

[PaulieDC]

Exactly. You WANT that wonderful raid drive, you just don't want the OS and apps on there the way Apple currently has it. Btw, once you install the OS, you can drag a lot of your programs from the raid drive to the Raptor, cool function of Macs. But that's why you need to get the Raptor going first with the OS, to make sure it all runs. I know, it seems daunting to redo the way Apple has it config'd, but remember that it's just a box of electronic parts that you can muck around with, lol!

[Guest]

OK PaulieDC Here we go:

Currently waiting on the WD VelociRaptor 600GB 10,000rpm drive. (Never thought I would say that, I now sound like a total tech geek)

The set up will be this:

Mac Pro 8 Core 2.4 GHz with Intel Xeon Processors

16GB RAM

NVIDIA Quadro 4000 Grpahics Card

Mac Pro RAID Card

Three 1TB SATA Drives 7200rpm in a RAID 5 Configuration for audio and video assets

One WD Veloci Raptor 600GB 10,000rpm drive for OS and Apps

One iOmega 2TB Firewire 800 external drive for back up and storage.

Oh, and one large debt on my credit card.

It looks like the set up will be easy. Just install new drive. Designate new drive as boot drive for OS and Apps. Go into disk utility and wipe the RAID clean of OS and Apps. Install all apps on WD. and I am off to the races with Pr Pro CS 5 for video and Logic Studio for audio.

I will let you know how this pans out after I get it set up.

Bill

[PaulieDC]

Dear Bill the Geek,

Methinks you have a dandy setup. Post back when you get it fired up!

Paulie

[Guest]

Paulie,

Thanks. You have been very helpful through all of this. I should get the drive tomorrow. What I would like to know in advance if possible is how I am going to set Pr up on this.

You say that all of my assets will be on my RAID and the OS and Apps on the Raptor. How do I get all of my assets on the RAID? I am sure that is done through Pr to have it writing and reading from there, but how do I do that. Also, I need to make sure that I am backing up the necessary files to an external drive so that I can work from two computers at two locations. I need to know what those files are so that I have seemless work flow between both.

Thank you so much and when I get better at all of this I hope I can return the favor for those here on the forum by providing some good info.

Bill

[FITB_STUDIOS]

Just a thought for you Billy, or was it Pauly. Crap I have lost track. Anyways who ever it was that is setting this up and is a wedding videographer. Here is my experience as I shoot weddings myself. My other screen name was Switch7 from much earlier in this thread. I am sure you read through my build. I couldn't log in with that name for some strange reason so I created a new account. Anyways.

If you plan to shoot a summer of wedding footage. 3 - 1Tb drives in raid 5 will not be enough storage for you. I started out with a similar set up but had 4 640's in raid 5 which gives less than 2Tb. The same you will have. It will fill up quickly. I shoot about 250GB of footage per wedding and if your shooting 10 - 12 weddings in a summer, well you do the math.

What I ended up doing is building an external Raid 5 array with the $300 Sans Digital stand alone external raid Box it has 5 slots, currently I have 4 2TB WD Blacks in there. (Not ideal raid drives I know, but cheap with tons of speed and storage) I now have over 5.45TB after all the conversions and no editing issues. Man that feels like highway robbery, 8tb labeled down to 5.45tb? Sheesh. But I digress. With all my footage stored externally, connected through eSata. I get all the benefits, plus enough storage to get me through a summer. And to top it off. If there is a fire in my building it is much easier to run away with a fully isolated raid box, over my massive computer. Thus keeping my business assets safe and project files safe.

On a side note I also have every wedding I shoot backed up on Individual WD 320Gb External USB Drives and kept off site in a safe deposit box. Peace of mind is worth the whopping $28 a year I pay for my Safe box. LOL

Something to think about.

[FITB_STUDIOS]

Ok I am on crack with my reply about your current set up. I lost track of where this thread went. Sorry about my references, but I will keep the post, as there is some good info there anyways for others.

[EstebanMartin]

i dont understand,

i have two discs 500gb x 2, the fastest should be used for the SO/pagefile and the other for media/project/cache ?

i used one for project/pagefile/exports and other for media.

what is the best config to use ?

[Guest]

I too am still a bit confused about this myself, but I am getting a bit closer. I think it really depends on how many drives you have at your disposal that will determine how you are going to set up Premire Pro.  According to Adobe by default the scratch disk files are stored where you save your project. But as as sequences become larger, the amount of disk space increases. Adobe recommends that you dedicate a hard disk or disks strictly for your media. This disk/disks should be separate from where you keep your OS, Apps and Project Files. If you have many disks at your disposal, you could dedicate a different disk for each kind of asset like captured video, captured audio, etc. But, you could just as easily have your media disk (the one separate from the OS/Apps and Project File) have folders in them to keep you assets organized. Adobe recommends that your fastest disk be the one for the scratch files that are accessing the media.

I was advised by PaulieDC to use a fast Raptor (10,00rpm) for my OS and apps and I guess this is where I will also keep my project files now as well. But the scratch disk will be my three 7200rpm disks set up in RAID 5. Adobe advises to use the fastest drives for the scratch. In my case, a 10,00rpm drive vs three drives in RAID 5 on a RAID card probably are pretty close in speed and I prefer to have the redundancy where my media files are located in order to provide ample back up.

[Guest]

PaulieDC and Wonderspark

,

Got everything set up. It was as easy as erasing the Volume and keeping the RAID in place. No problem. Main drive for OS, Apps and project files is the WD Raptor 600GB 10,000rpm

My RAID drive is where all the scratch is going to take place with all of my media files which I will organize on the RAID when I open each project.

I have to now install the drivers and CUDA for the NVIDIA Quadro 4000 and then drop her in place. I have an edit to work on this weekend so I will hit the ground running. I also decided to return the iOmega 2TB external drive Firewire 800 in exchange for a G-Tech 1TB External RAID drive that I will run in RAID 1 through eSATA (have to mod the Mac for that). I will use this drive for Time Machine and for storage from compoleted work done on the internal RAID.

Here is a question regarding this. Am I able to designate different locations for different projects. For example, I would want to use the external RAID 1 drive to work on projects that I know I will need to work on with other editors on other computers, so having that one designated for these projects would be nice so I can easily move it back and forth. The rest of my projects (the majority) will all be done internally.

Also, what files do I need to copy over from my internal RAID to my external storage so that I can store them and open them later if I need to. I do not want to junk up my internal RAID with a bunch of old footage from old projects as I move forward.

Thanks.

[PaulieDC]

Main drive for OS, Apps and project files is the WD Raptor 600GB 10,000rpm

My RAID drive is where all the scratch is going to take place with all of my media files which I will organize on the RAID when I open each project.

Given your hard drive arrangement, I think you've got it! If Adobe says it's OK to store Scratch and Footage on the same drive, then that works for me. It'll never hurt to have 29 separate drives, but we live in reality, not fantasy, lol. Your earlier post mentioned just having folders for organization as being the same as separate drives (maybe I read that wrong), but while organization is extremely important, you always gain the most speed having separate drives, just FYI. Imagine 6 different heads reading data and all of them streaming in concurrently at many megabytes per second. BUT, moot point, you have a good setup now, 10K OS/App/Project drive and a killer RAID for footage and Scratch. Well done. Much better than running it all off the RAID. AND, absolutely excellent move getting the external eSata RAID!

OK, your other question: yes, when you start a new project in Premiere, right there is where you'll designate where to store the project files. If you already have a folder on the ext RAID for projects, then just create your new folder for the individual projects on the fly at the Save step in Premiere when you open it. Simple, just like anything else you save. But, remember that previews and rendered video on a timeline is most likely stored in cache in a different location, so when you open the project on a different system, you may have to render again, or at least point to the footage. By the way, speaking of that, you're going to need your footage too, correct? Will you just copy the footage onto the ext RAID and then copy it onto the other computer when you get there? That seems easiest. Remember, your project and all of its files will be on that ext RAID, but the FOOTAGE won't be, being it's on your internal RAID. The timeline just points to its location. So you'll want to copy that separately either onto that ext RAID if there's room or sme other ext drive, just for transport's sake. Don't worry when you get to the second location, you can dump the footage on that system whever you want. When you start Premiere on the second PC it's not going to find the footage because the location is stored as on your Mac. But not to worry, Pr will pop up a dialog and say "Hey! Where's your footage??". Actually it'll ask for the FIRST file it doesn't find. You just navigate to the file it wants, and it will automatically find the remaining ones. NOW, if you're proficient at offline editing and you already KNOW all this stuff, please disregard! I just want to make sure you have all your bases covered.

[Guest]

For those projects that I will do "offline" I will be storing the media for them on the external drive as

well. The idea is to hook that external drive up to the second computer and have Premiere work from that drive alone

Will that work? I thnk it should. So all I need is the project file, and original media correct? Or is there some other files I will need as well?

[PaulieDC]

I think that'll work! There will be several files that Premiere will create, but they will all get auto-stored in whatever folder you create the project in, so as long as you have the project FOLDER, you'll have all you need. Go for it! If it doesn't work the we'll figure it out.

[Guest]

Paulie,

Should that project FOLDER be kept on the OS drive. It seems Adobe states that the OS, APPS, and Project files be stored separate from the scratch disk??

[wonderspark]

I know you didn't ask me, but for the record, I keep my OS and Adobe applications on the OS drive, and all remaining files / folders (including the project files and all scratch folders) on the asset drive, which in my case is a 3-disk RAID 0.  I have no problems with data bottlenecks.

[wonderspark]

The only potential problem I see is the external drive's file system type compatibility, going back and forth from Mac to PC.  The PC will almost certainly be NTFS, and Macs are usually HFS+ (aka Mac OS Extended), and if the two computers aren't both able to read the file system type on the external drive, one will not recognize the drive.

I handle this by installing Tuxera NTFS on my Mac, and format the external drive to NTFS via Disk Utility, selecting Tuxera NTFS.  One could also just use FAT32, but my video files are usually over 2GB in size, and FAT32 won't allow for a file over 2GB.  It's important to note that even with Tuxera on my Mac, it's not a perfect system.  For example, on a Toshiba external drive my friend uses, I have to reboot my system with the drive attached and turned on in order for it to be recognized, and when I transfer files to it, the displayed space available will not update until I reboot my system AGAIN.  Annoying, but it works fine otherwise.  This is using the ultra-fast SATA connection.  It works normally if I use the other port on the external drive... super-slow USB 2.0.  It's yet another thing to think about.

ExFAT is supposed to be the magic pill to fix all this compatibility, but I haven't had success with that yet.

You mentioned modifying the Mac for eSATA... OWC sells eSATA cards for $49.  Link: http://eshop.macsales.com/item/Newer%20Technology/MXPCIE6GS2/

I have this card in my Mac Pro, and I love it.  They also sell a version of this card for $70 that supports RAID and port multipliers... link: http://eshop.macsales.com/item/Newer%20Technology/MXPCIE6GRS/

I actually wish I had that second version with RAID support.

[Guest]

Wonderspark,

Thanks for the tip. I am going for the $49.00 one you linked to. I like this better than just the cable that would take one of my optical drives away. I think I am getting the right one. Even though I am running the G-Tech RAID mini in RAID 1,  it does not have a port multiplier, so I guess I am fine with just the $49.00 one. (I hope). THe $70.00 one is for a RAID set up but with systems that use port multipliers and usually several drives. The RAID mini is two drives in RAID 1.

[Rob_de_Boer]

A few weeks ago I got my new system. I took a good look at my budget and based my HDD setup on the recommendation in the first post of this thread.

I have done the PPBM5 benchmark twice. Once with my system at stock speeds, and once with an overclock on CPU and memory. The overclocked system runs every test faster than the system at stock speeds, except for the disk intensive AVI encoding, which shows a 43 second worse performance! On my total benchmarktime it even more than cancels out the gain I get from the better scores on the other tests. Both runs were done with version 5.0.3. Bill Gehrke already let me know that 5.0.3 for some reason has worse disk io performance than 5.0.2, but considering I ran both times in the same version that should not be an explanation.

The reason I post this in this thread is that I'm wondering if performance should increase if I change my HDD setup. Which currently is as follows:

C: 120GB SSD: OS and programs

D: 1TB 7200 SATA: Adobe Media Cache and Windows Pagefile

E: 1TB 7200 SATA: Footage, other media and Adobe project files

F: 1TB 7200 SATA: Adobe Exports and Adobe Preview files

All 1TB HDD's are identical WD Caviar blacks,

This basically follows Harm's guideline for a 4 disk setup. However, other options are mentioned for a 4 disk setup. Somewhere in this thread they are named 4a and 4b, and both involve a RAID 0 setup of the Export disk (F in my case) and either the Pagefile disk (D in my case) or the project disk (E in my case).

My question is simple. Will setting up a RAID 0 in either of those two options going to (substantially) increase my performance? If so, which option would increase my performance most?

I work predominantly with AVCHD files (1080p 50fps) and some CPU intensive plugins.  

Don't know if this is relevant, but my other specs are:

Core i7 950 (3.8 GHz OC)

12 GB RAM -1600 (1680MHz OC)

Gigabyte 460 GTX (815 MHz factory OC)

Windows 7

Other than that I'm really clueless about an explanation for the weird results that I mentioned in the first part of my post. If anyone has any ideas they're more than welcome to share it. I haven't changed anything on the multipliers for the uncore clock and QPI, so they have both gone up with the bCLK increase. So that is probably no explanation.

Slightly different, but maybe even more important question: Does it matter for my use of premiere pro? Considering I'm only working with highly compressed AVCHD files (max around 27 Mb/s).

Woops, that's already three big questions in my first post ever to this forum. Sorry about that. Didn't mean to come barging in like that... I also sent Bill Gehrke an e-mail about this, but I figured I should stop harassing him with all these questions and put em where they're supposed to go, which is here.

[Rob_de_Boer]

Hi again,

Is there anyone who has any thoughts on any of the questions I have regarding:

1. my disk setup (will raid improve my performance?)

2. and the disk io performance on the ppbm5 benchmark with and without an overclocked system?

3. the importance of disk performance for my specific use of premiere pro (mainly avchd files)

Please see my post of january 10th for a more elaborate explanation. Any input is very much appreciated!

Regards,

Rob

[Harm_Millaard]

Where is the post?

[PaulieDC]

Scroll up a few feet, Harm...