Generic Guideline for Disk Setup

[Harm_Millaard]

There have been many questions about how to set up your disks.

Where do I put my media cache files, where the page file, and what about my preview files?

All these allocations can be set in PR, so I made this overview to help you find some settings that may be beneficial. It is not a law to do it like this, it is a generic approach that would suit many users, but depending on source material, workflow and backup possibilities, it is not unthinkable you need to deviate from this approach in your individual case.

The reasoning behind this overview is that you want to distibute disk access across as many disks as possible and get the best performance.

Look for yourself:

I hope this helps to remove doubts you may have had about your setup or to find a setup that improves performance.

[PaulieDC]

AH, good, explains that!

[Alex_Curry]

Thanks Harm, I have the 12 port version like you recommended. Is the 667 a better choice, what is the difference?

[Harm_Millaard]

Alex,

You should try to find a Kingston KVR667D2E5/2G module. That works great. I would be wary of other modules, because the Areca card is rather picky.

[UlfLaursen]

Harm, or others

Do you have any experience with the Seagate 'momentus' discs, and possible in RAID, or any comment on them?

http://www.seagate.com/www/en-us/products/laptops/laptop-hdd

Thanks.

/Ulf

[stenapp]

Alex, I feel your pain.

Whether or not to include the OS drive in the RAID depends on what the RAID is for.  However, in the context of the rest of this thread, I am going to assume you are building a large RAID for storage of footage as you work on it.  In that case, keep the OS drive separate.  Install the OS drive on the motherboard, and the raid drives on the areca card only.  The areca card I had my tech use had a limit of 4 drives.  I ended up with three drives in a RAID 3 with a hot spare also on the card.  That shows up as 1.81 TB for the four drives, which is about right, considering one drive is the spare, one is for parity info, and the other two lose ~28 GB in the format. My OS drive is another 640 Gb.

I don't know the specs on your card, but these are things to watch for.

BTW my tech warned me that formatting the RAID took him nearly 20 hours, but that was necessary in order to get it to show capacity.

[PaulieDC]

Excellent guide Harm. I came up with a setup that I thought would work (and it does so far), and it turns out to be your 4-disk setup (technically 5 because my asset drive is RAID0), so you confirmed that I'm on the right track.

Thanks!

[Marcus_Murphy]

Planning to get an Areca ARC-1210 with four 600GB Velociraptors in RAID 0 (considering R3, but I can set up daily back ups, not too worried about the data). Would there be a RAID controller better suited for this without increasing price too much (~$300)? All reviews I read point to Areca and the next best controller I see is the ARC-1231ML at $600. I have 4 other Western Digital Black 1TB's that are not in RAID currently (don't really plan to either) and an 80GB SSD as a boot drive. Thanks!

[Switch7]

Why would you waste all that money on Velociraptors when (correct me if I am wrong) but the WD 2TB Blacks are faster and much cheaper?

[PaulieDC]

WD Blacks are 7,200RPM, Raptors are 10,000RPM. But also, Blacks are not always a good RAID choice, they have time-out circuitry that corrects for erros when standalone, but can cause RAID to drop. Hence, the WD RE3 drives, made for RAID, no timeout circuitry.

[davidbeisner2010]

Thanks, Harm, for your expertise! I've read this post and your post on general storage setup for editing systems, and I've got some questions...

First, I've got a TON of data, much of which I could probably archive to clean things up. Here's what I've got right now, and maybe you can give me some recommendations for fixing things so it'll run better (my PPBM5 shows significant slowdown in disc-land). FWIW, all internal drives are 7200RPM SATA2.

I have a 250GB WD Enterprise Storage drive for my OS/Programs, and I think it holds the pagefile... (though I have 12GB of Ram, so I'm not sure how often it's used). This one is about 50% full.

I have a 500GB WD Enterprise Storage drive which holds all my graphics, photos, and audio, along with some other generic data files (think documents, PDFs, etc. here) This one is about 60% full.

I have a 1.0 TB WD Caviar Black for my Projects/Scratch/Previews disc. This one is about 40% full.

I have a 2TB RAID-0 (made from 2x 1.0TB WD Enterprise Storage drives) that holds all my data in whatever format it came in on (most of it is direct capture from HDV in Premiere, but there's a good mix of AVCHD, R3D, and various SD stuff, too). This one is about 95% full.

I also have a 4TB RAID-0 WD backup drive that's a little over 50% full, and maintains a complete backup of all the drives in my system, minus OS and Programs. (FW800)

Then I have a second 2TB single disc drive that I use to maintain an off-site backup of whatever projects I'm currently working on, plus important stuff like photos and graphics. This one is about 30% full. (FW800)

Then I have a third 1TB single disc drive that is a general backup drive and usually remains onsite, and is about 80% full. (USB 2.0).

Ok, so there's where things are right now. Based on what I've read, I'm thinking that I might be better off doing the following:

250GB for OS/Programs

500GB for Previews/Scratch/Pagefile

3x 1.0TB drives (no RAID) for my projects/media OR 1.0TB in RAID3/5 for projects/media (but I'll have to figure a way to archive a bunch of stuff... Bluray?)

I've got room in my system to add one more drive, and I can probably get approval for purchase of one more drive if I asked my boss--so if I got a 2TB drive, could I do a 2.0TB RAID using two of the 1.0TB drives for striping and the 2TB for parity, leaving me with an extra 1.0TB? Or should I get a 1TB drive and do a 2.0TB Raid with two drives for striping and two for parity? I'm really not up on this RAID stuff, despite reading your article on it and Wiki...

Also, could I separate my current RAID-0 and not lose the data on it?

And just FYI, I'm using onboard RAID (using the ASUS P6T6 Server mobo), and purchase of a RAID controller is out of the question right now...

I primarily edit HDV/AVCHD, but I have a little bit of RED 4k and I'm hoping to upgrade my HDV camera to the JVC GY-HM700 soon.

Thanks for your advice and expertise!

David

[Draal Lala]

I'm not sure I understand your recommendations for four disks. "D: or E: and F: can be run in Raid0".  Does that mean D: and F: or E: and F:? Or does it mean D: or E: and F:? That's not the same. Hope you understand my confusion.

I run automatically in this four disk configuration because the SSD I would like to buy (C300) has excellent reads but slow writes. So it is good for OS/Programs and bad for the page file.

[Harm_Millaard]

The first one:

Does that mean D: and F: or E: and F:?

Sorry for not being clearer.

[Draal Lala]

I still don't understand

[stenapp]

I believe he is saying

        D: and F:

[Harm_Millaard]

1. D & F

or

2. E & F

[Draal Lala]

Thanks.  So you can actuallysplit this 4-disk-option in two options: 4a) C:, D:, E/F combined; 4b) C:, E:, D/F combined.

E: and F: seems to make more sense though.

From your table I conclude on of the most important Disk Setup Laws: to separate the pagefile from media and projects.

[Another Photographer]

Very nice chart!

I'm hoping you can provide some specific guidance for my situation.

I edit a lot of AVCHD and some MJPEG (all from DSLRs, 720p or 1080p).

I currently have two drives:

C: (Seagate 7200 RPM, 750GB) I use this for OS, Programs, Pagefile, Projects, Previews, Exports.  I am not sure what Media Cache is, but I'm sure it's here as well.

D: (Western Digitial 2TB Green, slower than 7200RPM for sure, feels super slow to me, but haven't benchmarked).  I use this just for raw footage exclusively.

Would you set this up any differently?

If I were to add one more drive, what makes the most sense?  SSD for the OS/Programs/Pagefile?  Should I just move the slow 2TB Green into an external enclosure and use for backups?

Message was edited by: Another Photographer

[Harm_Millaard]

Get two new 7200 RPM SATA disks. Your current drive C and the new disks as D and E and use your current Green disk as F for backups. Then apply the three disk situation, C, D and E. You could consider using the F drive for exports with its huge size as well.

[Another Photographer]

Thank you Harm.

In the 3-disk scenario, what should be the size of E:?

[Harm_Millaard]

SATA disk start to slow down when you reach a fill rate of around 65 - 70 %, so if you expect to have say 400 GB of data, the size should be 640 Gb, if you expect around 600 GB of data, get a 1 TB drive, if you expect 1.3 TB of space get a 2 TB drive, if you need more use two disks in a raid0.

[Another Photographer]

Thanks Harm.  I was wondering how large "Previews and Media Cache" would be. My exports won't be huge, so if Previews and Media Cache were not large either, drive E: in the 3-drive setup would be relatively small (maybe 500GB?).

[Harm_Millaard]

Get the largest you can get. A 500 GB is almost as expensive as a 1 TB. The difference in price is negligent, especially in comparison to CS5.

[atpco8214]

Hi Harm,

Thank you for all your posts. I have learned a lot by reading them. I am in the process of setting up a system for video production with CS5. I plan to purchase 4-10k RPM disks and had originally thought to set them up in a RAID 1+0 but I notice that you say repeatedly not to set up a RAID array with less than 5 disks.  I am concerned that adobe requirements for CS5 specifically say that RAID0 is required for uncompressed video. Is that not the case and with that in mind, do you recommend that I set them up as single drives? Thanks.

[Harm_Millaard]

A normal data rate for compressed 4:2:0 HD material is somewhere in the range of 25 Mbps, easy to handle for modern disks. Even with 2 or 3 video tracks. But uncompressed HD-SDI ingest in 4:2:2 HD format may go up to 1.485 Gbps, which means a sustained transfer rate of more than 200 MB/s per video track. In practice that would require a 3 or 4 disk raid0, depending on the fill rate. Imagine what would happen if you have four video tracks, that can add up to a sustained transfer rate of around 800 MB/s.

For background: http://en.wikipedia.org/wiki/SMPTE_292M

Striped raids entail the risk of data loss but increase performance.

Mirrored raids double the cost without performance benefit but with added security.

Parity raids combine the benfits, more performance and more security, but at less cost than mirrored arrays.

[Another Photographer]

Would you recommend using USB 3 in place of SATA or eSATA drives?