• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
Locked
6

Generic Guideline for Disk Setup

LEGEND ,
Jun 18, 2010 Jun 18, 2010

Copy link to clipboard

Copied

There have been many questions about how to set up your disks.

Where do I put my media cache files, where the page file, and what about my preview files?

All these allocations can be set in PR, so I made this overview to help you find some settings that may be beneficial. It is not a law to do it like this, it is a generic approach that would suit many users, but depending on source material, workflow and backup possibilities, it is not unthinkable you need to deviate from this approach in your individual case.

The reasoning behind this overview is that you want to distibute disk access across as many disks as possible and get the best performance.

Look for yourself:

Guideline Disks.jpg

I hope this helps to remove doubts you may have had about your setup or to find a setup that improves performance.

Views

353.6K

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
replies 425 Replies 425
LEGEND ,
Dec 29, 2010 Dec 29, 2010

Copy link to clipboard

Copied

Do you know what USB stands for?

Universally Serious Backups = USB. That is all USB drives can be used for. Even with the much touted theoretical bandwidth of USB3, the USB3 disks are still around 100+ % slower than SATA and there are no raid controllers for USB drives in existence. It is all marketing hype. USB2 could not achieve more than 20-25 MB/s and it looks like USB3 will be hard pressed to achieve more than three times that performance, still lagging way behind SATA/SAS connections.

From early benchmark results I have read, the speed of USB3 drives was comparable to FW-800 drives. Things may have  improved somewhat since those early tests, but still no match for SATA/eSATA.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Dec 29, 2010 Dec 29, 2010

Copy link to clipboard

Copied

I realize that USB 3.0 is slower than RAID, but is a 7200RPM drive, which is limited to say 125 MB/sec, going to be slower on USB 3 than SATA?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Dec 29, 2010 Dec 29, 2010

Copy link to clipboard

Copied

So, to handle the uncompressed format I should set them up with one as the primary boot drive and the other three in a RAID0?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Jan 03, 2011 Jan 03, 2011

Copy link to clipboard

Copied

Thanks Harm.

You said "Get two new 7200 RPM SATA disks. Your current drive C and the new disks as D and E and use your current Green disk as F for backups. "

It looks like I can squeeze only a single new drive out of my budgeting department.  It will be a Caviar Black.

I'm thinking the Caviar Black (D:) should have Media and Projects, and Caviar Green (E:) should have Exports.  C: will be for OS, Programs, and Pagefile.  Which drive should Previews and Media Cache be on?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Explorer ,
Dec 23, 2010 Dec 23, 2010

Copy link to clipboard

Copied

New to all this, but I've been reading for days now.  Thanks for all your help.......... One source of confusion.

Yesterday I found this here (somewhere)   C: OS & Programs

                                                             D: Projects & Scratch

                                                             E: Media

Now, at the begining of this thread I see your table.    Does the difference matter??

         Thanks,  Tom B.             

                               

image_jpeg.jpg

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Contributor ,
Dec 30, 2010 Dec 30, 2010

Copy link to clipboard

Copied

Thank you, Harm for an excellent guide!

I have the following setup:

OS: Win 7 64 - Ultimate

MB: Gigabyte GA-EX58-Extreme

CPU: Intel Core i7 975X

RAM: 24GB, (6x4 GB Mushkin DDR3 1333)

GFX: Nvidia GTX285

RAID: 3ware 9750-4i

Single Drives:

OS Drive: 7,200 SATA 500GB

Prog Drive: 7,200 SATA 500GB

Page file and Misc.: 7,200 e-SATA 1TB

RAIDS:

Projects, Media, Render etc.: 4 x 1TB SAS2 RAID-5

Backups: 4 x 1TB SATA RAID-5 (e-SATA)

I have one question; My read speed from the SAS raid averages about 400MB/s which I think is OK, but the write speed is about 80-100 MB/s. Is that normal, is that a result of the nature of RAID-5 over 4 disks?

Thanks,

Magnus

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Dec 30, 2010 Dec 30, 2010

Copy link to clipboard

Copied

Magnus,

That figure seems very low. With a 4 disk raid5, even on a 3Ware controller, I would have expected that to be in about or in excess of 200 MB/s.

What processes do you have running? Please post a screen shot from Process Explorer, not Task Manager. And post this in a new thread to avoid contamination.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Contributor ,
Dec 30, 2010 Dec 30, 2010

Copy link to clipboard

Copied

Thanks Harm, will do once I get home!

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guru ,
Dec 30, 2010 Dec 30, 2010

Copy link to clipboard

Copied

Set your write policy for the raid volume properties to "Always Write Back"

Eric

ADK

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Contributor ,
Dec 30, 2010 Dec 30, 2010

Copy link to clipboard

Copied

Thanks, Eric!

I currently have:

Write Cache: Enabled

Read Cache: Intelligent

Auto Verify: x (checked)

Overwrite ECC: - (unchecked)

Queuing: x (checked)

I was thinking that maybe the queuing should be unchecked for SAS drives?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guru ,
Dec 30, 2010 Dec 30, 2010

Copy link to clipboard

Copied

Are you using the Raid console management in windows or the bios raid management?

Use the one in Windows and look for a write policy setting under your raid volume properties. If you can't find it contact the manufacturer and ask them where that setting is at.

Windows 7 will continue to throttle the drive performance during write operations for data integrity until you change that setting.

Eric

ADK

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Contributor ,
Dec 30, 2010 Dec 30, 2010

Copy link to clipboard

Copied

For  my controller (9750-4i) Write Back is called Write Cache, I believe. If I uncheck that I assume it would act as Write Through because write speed drops to 10-12 MB/s!

I am using the Web Interface through Firefox.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guru ,
Dec 30, 2010 Dec 30, 2010

Copy link to clipboard

Copied

I would contact the manufacturer. Because the write speed your getting still sounds like the write policy is incorrect. The Cache settings are normally strictly tied to controller and drive cache and not policy settings which are separate. Disabling the cache for writing would still enable Controller cache for reading. Most controllers just give you an option to enable or disable controller cache.  It looks like they split those up so you can decide which instructions get cached. That is not the same thing as write policy. It's up to you but I would contact them.

Eric

ADK

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Contributor ,
Dec 30, 2010 Dec 30, 2010

Copy link to clipboard

Copied

Thank you, Eric, for clarifying!

I will certainly contact 3ware/LSI and see if they can direct me in the right direction!

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Contributor ,
Dec 30, 2010 Dec 30, 2010

Copy link to clipboard

Copied

I contacted 3ware support and they had me update the firmware for my Seagate Constellation drives and then test the throughput using IOmeter.

These are the results I get for write throughput now: 399MB/s.

IOmeter results.jpg

IOmeter.jpg

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guest
Dec 31, 2010 Dec 31, 2010

Copy link to clipboard

Copied

This is a great thread. Thanks for all the info. I am hoping that I can get some answers here.  I just got my new Mac Pro last night and had it set up into a RAID 5 configuration mostly because I was not sure what I needed and the Apple guy insisted this is what I needed. I have a Mac Pro running Snow Leopard 10.6.5 three internal 1 TB hard drives with one external 2TB iOmega Hard Drive Firewire 800. I will be using this for Adobe Premiere Pro CS5 and Logic Studio.

I do primarily wedding video and some other small videos (documentaries) all on uncompressed media from Canon 5D and Canon 7D as well as video captures from other sources. I want to optimize the system. I am thinking that the RAID 5 I set up  from the install disc to get all three drives in the RAID set is not the best use of this set up. I am thinking I need to put the OS and apps on one drive and then have the other two in a RAID 0 and then use my external for storage of video files and music files. But, don't know where to keep the project files and where I should do the scratch, etc.  Or should I get a fourth drive, and re-do my RAID 5 to include that drive, if I do, how do I dedicate the specific drives to house the specific tasks. For example how would I get just one drive to do the OS? I understand I can dedicate the rest of the functions of the drives in the RAID set via Pr Pro after the RAID is put together. 

Please provide some help if you can. It makes no sense for me to start work in this configuration since if I need to change it later, there will be much reworking of data. Figured I better get this sorted out asap so I can get up and running in the right way.

Thanks

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Engaged ,
Dec 31, 2010 Dec 31, 2010

Copy link to clipboard

Copied

Hi Billy,

Well, you certainly have the horsepower you need, but it needs a little configuring. I shoot 5D2 footage and use Premiere Pro as well, and there are a few things to know to make this easy so it is done right the first time.

We'll discuss your Mac in a second, but first you should know that you are going to need to pull off every trick in the book to get the footage to edit smoothly. You may know a lot of this, I'm just including it so it's all in one basket (and to benefit another reader possibly). The H.264 QT format that comes out of the 5D Mark II and 7D is a good compressed format for finished video, but when we attempt to edit it, frustration can occur. You aren't editing uncompressed video by any means, if it was it would be 4K footage being that the 5D2 sensor is 5616x3744. Wouldn't that be awesome if we COULD get that out of it??

To be clearer on this, the 5D2 compresses the image down to 1920x1200 in H.264 (H.264 for video is the same thing as shooting a still image in JPEG format... we have RAW for stills but not for video YET in a 5D2). The compression used by Canon stuffs the video down into an 8-bit 4:2:0 color space which is more lossy than I would prefer, but I imagine Canon did that to make the footage as manageable as can be. In fact, with Premiere Pro CS4, it was uneditable, it just froze Premiere. To fix this, we all bought Cineform's NeoScene which converts the 5D2 footage up to a 4:2:2 10-bit file (still 8-bit but now with headroom for color grading, etc) and converts it into an .AVI file so it'll run smoothly in Windows. They do have a Mac version also, and you can buy it at VideoGuys.com for $99. ANYWAY, that's how we were able to edit in CS4. Now that CS5 is out, Adobe went to great lengths to accomodate the DSLR world by adding the right presets, etc, but most importantly by adding the MPE for all editors to leverage for all types of formats. not just DSLR vids. Now, the Mercury Playback Engine helps greatly, but as soon as you stack some effects onto the footage, it can still get jumpy. To this day I still convert everything with NeoScene because in conjuction with the MPE it's even that much better. Premiere Pro CS5 does leverage the CUDA technology in the video cards to help power that MPE, but that does mean having a compatible Nvidia graphics card, not sure if you have that in your Mac. I know there's a GTX 285 for the Mac that would work, you would have to research other models such as the uber-expensive Quadro series. I don't think there's a card in the 400 or 500 Nvidia series for Mac yet. But that may be a good reason to get NeoScene anyway.

OK, what on earth did all that have to do with your Mac's hard drives? Glad you asked! Once you ensure you have the best converted file available to use for editing (by doing all that above), you then want to place that file on your fastest drive which would be your RAID5 config. You need read speed like crazy which RAID5 will give you. You also get redundancy with that. I use RAID0 but I always back up my footage first so if RAID0 drops, I can quickly recover. I realize that doesn't help me halfway through a 6 hour encoding session, lol. ANYWAY, there's one issue with your setup: The OS and programs are also on that RAID5, correct? That's what you don't want. It'd be great if you could install a 10K RPM 300GB WD Raptor and put your OS and the Adobe Suite on that drive and save your RAID5 for your assets. I would go one step further and add one more drive to write your output video to and to also use for the Adobe Scratch Disks. Don't use partitions, and by all means don't use the external drives, FireWire is way too slow. The only external drive you would want is eSata, but really you want your footage on that RAID5 setup. I know, this all sounds kookie, you probably just wanted to get the powerhouse Mac with RAID and be done with it. That'd be fine for gaming, but for video editing you need the right setup, it will make your life much much easier.

BTW, do you have at least 12GB of RAM? We love Adobe software, but it eats ram like a cheetah eats gazelles...

OK, that's all I have. Whatever you choose to do, just make sure your footage is on that RAID5 rig, even if it shares it with the OS. Worst case, you could always buy just one additional internal drive and put all the Adobe software on it, and leave the OS on the RAID5 with your assets, because the Mac OS loads mostly into memory anyway.

Recap: Your best config if it was possible:

  • 10K RPM Raptor for OS and Adobe Suite
  • RAID5 for all assets
  • 3rd drive for Project files and output
  • 4th drive for Scratch Disks and whatever other stuff you want on your PC (Photos, etc). Keep that stuff isolated so they're not in the way of your Assets or OS. The Scratch Disk is a great place to do that.
  • Use your FireWire external for backups.

This is pretty much the setup I use on the PC I built and it works well.

There it is! If I'm off track, somebody pipe in, let's get Billy up and running the right way the first time!

Message was edited by: PaulieDC... fixing typos!

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Dec 31, 2010 Dec 31, 2010

Copy link to clipboard

Copied

Thanks for the great info!

In a three-drive set-up as described in the original post, which would drive would need to be the fastest?  Which could be your slowest?  (Assuming all three are 7200RPM SATA drvies of varying speeds/ages)?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Engaged ,
Jan 01, 2011 Jan 01, 2011

Copy link to clipboard

Copied

Another Photographer, your assets need to be on the fastest drive, you need lots of READ speed when editing. Your slowest can be the scratch disk drive. So in the most minimal setup, a Raptor for OS & Adobe Suite (don't need to RAID it really), then RAID0 or RAID5 for assets, and a good 7200rpm drive for scratch. If you do RAID0 for assets, you'd better have backup copies of everything on that drive stuck out on some external drive. Really, adding the third drive to make RAID5 is smartest so you get redundancy. That one extra drive is peanuts compared to what we spend on these rigs, lol!

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guest
Jan 01, 2011 Jan 01, 2011

Copy link to clipboard

Copied

Another Photographer, If you do RAID 5 with only 3 drives, it will distribute the OS across the set, which is my problem currently. As far as the best set up, according to Paulie and some others, is that one of the drives in the set needs to have the OS. In a three drive RAID 5 all drives are in the RAID set with the opporating system. That is how I understand it, I may be wrong as I too am still trying to figure this out. I am getting a fourth drive to do what is being suggested here, but don't know how to do that. Waiting to hear how this should be done.

Thanks

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Jan 01, 2011 Jan 01, 2011

Copy link to clipboard

Copied

Thanks guys!  I still have a lot to learn.  Two questions/observations:

(1) I need to check whether my RAID controller allows me to attach both RAID and non-RAID devices at the same time.

(2) When I edit AVCHD, the files are only 1 or 2 Meg/sec.  I'm not seeing why I need RAID for such low-bandwidth files.  Please advise.

Thanks again!

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Engaged ,
Jan 01, 2011 Jan 01, 2011

Copy link to clipboard

Copied

(1) I need to check whether my RAID controller allows me to attach both RAID and non-RAID devices at the same time. I have no idea if RAID cards do that, but I would say for optimal performance, you probably want ONLY your RAID set on that card.

(2) When I edit AVCHD, the files are only 1 or 2 Meg/sec.  I'm not seeing why I need RAID for such low-bandwidth files.  Please advise. I don't work with AVCHD all that much but I've heard it's a fiesty format to edit. It may use a more complex codec algorithm that takes up CPU, but don't quote me on that! Others will need to pipe in and answer that one. But I would say that your interests and formats may widen in the near future so it's better to have your RAIDed editing rig already configured and running, you never know what may happen...

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Jan 04, 2011 Jan 04, 2011

Copy link to clipboard

Copied

I've just ordered components to build a video editing rig and I've enjoyed reading all the good suggestions on this forum.

My question concerns where the fastest drives shoud go.  Paulie mentioned that the scratch disk could go on your slowest drive, but this seems counter to what Adobe suggests.

Here is what I will be assembling:

ASUS P6X58D Premium MB

Corsair 850HX Power Supply

Intel Core i7 950 w/ CoolerMaster CPU cooler

EVGA GTX-470

G.Skill 24GB RAM Memory

Win 7 Pro, Adobe Production Premium and Lightroom

CoolerMaster Case

For hard drives I'll be primarily using these:

Crucial 64GB SSD SATA III

OCZ 60GB SSD

WD 1TB 7200RPM SATA III

WD 2TB Green SATA

Various older SATA drives for program files, storage or possibly a RAID array

External USB Drive(s) for storage

I am not into overclocking as I think it puts too much of a burden on the system heat-wise with not much real-world performance gains.

I just want a stable system that will last a long time.

My initial plans are to use the Crucial SSD as my system drive (C:) and the OCZ SSD as a cache or scratch drive.

The WD SATA III drive will hold my videos to be edited.

I figure my hundreds of thousands of photos can go on any of the slower drives.

Any suggestions as to how to set things up will be appreciated.

My main question is what to put where?  How best to use my faster drives, which I assume will be the SSDs and the 6GB/s SATA III drives.

Harm suggested a static cache file equal to 1.5x the size of your memory unless you have lots of memory.  What size would work best with 24GB of RAM?

Being new to Windows 7 I'll also appreciate advice on exactly how to direct things to other drives rather than everything getting piled up on the C: drive as Windows XP likes to do.

Thanks in advance.

-Bret

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jan 04, 2011 Jan 04, 2011

Copy link to clipboard

Copied

1. Crucial: OS & programs

2. OCZ: pagefile, around 12 GB static + media cache database

3. WD SATAIII: media, projects and media cache

4. Green disk: exports, auto save

5. USB: backups

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Jan 04, 2011 Jan 04, 2011

Copy link to clipboard

Copied

Please let me know if there is a better way to configure these drives (especially the first three):

(1) 750GB Seagate SATAIII 7200RPM: OS, Programs, Pagefile, Media Cache.

(2) 2TB Caviar Black SATAIII 7200RPM: Media, Projects, Previews.

(3) 2TB Caviar Green: Exports

--------------------------------

(4) USB: backups

Thanks in advance.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines