Storage rules for an editing rig. Some basics.

LEGEND ,
May 03, 2009 May 03, 2009

Copy link to clipboard

Copied

How do you set up your editing machine in terms of disks for maximum performance and reliability? (SSD's are left out here.)

This is a question that often arises and all too often one sees that initial settings are really suboptimal. These rules are intended to help you decide how to setup your disks to get the best response times. Of course the only disks in an editing machine must be 7200 RPM types or faster. No GREEN disks at all.

Rule 1: NEVER partition a disk. You may ask why? First of all, it does not increase disk space, it just allocates the space differently. However, the major drawback is that for a partitioned disk the OS must first access a partition table at the beginning of the disk for all accesses to the disk, thus requiring the heads to move to the beginning of the disk, then when it has gotten the partition info move to the designated area on the disk and perform the requested action. This means much more wear-and-tear on the mechanics of the disk, slower speeds and more overhead for the OS, all reducing efficiency.

Rule 2: Avoid using USB drives, since they are the slowest on the market. Do not be tricked by the alleged bandwidth of USB 2.0 advertisements, because is just is not true and remember that the alleged bandwidth is shared by all USB devices, so if you have a USB mouse, keyboard, printer, card reader or whatever, they all share the bandwidth. Stick to SCSI or SATA disks or e-SATA. If needed, you can use Firewire-800 or even Firewire-400 disks, but they are really more suited for backups than for editing.

Rule 3: Use at least 3 different physical disks on an editing machine, one for OS/programs, one for media and one for pagefile/scratch/renders. Even on a notebook with one internal drive it is easy to accomplish this by using a dual e-SATA to Express card connector. That gives you an additional two e-SATA connections for external disks.

Rule 4: Spread disk access across as many disks as you have. If you have OS & programs on disk C:, set your pagefile on another disk. Also set your pagefile to a fixed size, preferably somewhere around 1.5 times your physical memory.

Rule 5: Turn off index search and compression. Both will cause severe performance hits if you leave them on.

Rule 6: If the fill rate on any of your SATA disks goes over 60-70% it is time to get a larger or an additional disk.

Rule 7: Perform regular defrags on all of your disks. For instance, you can schedule this daily during your lunch break.

Rule 8: Keep your disks cool by using adequate airflow by means of additional fans if needed. You can use SMART to monitor disk temperatures, which should be under 35 degrees C at all times and normally hover around 20-24 C, at least in a properly cooled system.

Rule 9: If people want raid, the cheapest way is to use the on-board IHCR or Marvell chip, but it places a relatively high burden on the CPU. The best way is a hardware controller card, preferably based on the IOP348 chip. Areca ARC and ADAPTEC come to mind. 3Ware uses it's own chipset and though not bad, they are not in the same league as the other two. Promise and the like in the budget range are no good and a complete waste of money. Expect to spend around $ 800 plus for a good controller with 12 connectors internally and 4 e-SATA connectors. Important to consider in a purchasing decision is whether the on-board cache memory can be expanded from the regular 256/512 MB to 2 or even 4 GB. Be aware that 2 GB cache can be relatively cheap, but the 4 GB version extremely costly ($ 30 versus $ 300). For safety reasons it is advisable to include a battery backup module (BBM).

Rule 10: If you can easily replace the data in case of disk failure (like rendered files), go ahead and use raid0, but if you want any protection against data loss, use raid 3/5/6/10/30/50. For further protection you can use hot spares, diminishing downtime and performance degradation.


In general when you get a new disk, pay close attention to any rattling noise, do perform regular disk checks, and in case of doubt about reliability, exchange the disk under guarantee. Often a new disk will fail in the first three months. If they survive that period, most of the disks will survive for the next couple of years. If you use a lot of internal disks like I do (17), set staggered spin-up to around 1 second to lessen the burden on the PSU and improve stability.

Hope this helps to answer some basic questions. If not, let me know. Further enhancements and suggestions are welcome.

Views

106.0K

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
replies 143 Replies 143
Guide ,
Mar 14, 2012 Mar 14, 2012

Copy link to clipboard

Copied

Claire,

Sounds like you have a decent cpu, motherboard, and some drives to work with, and your video card is currently your weakest link.

Regarding the Greens, I would never suggest anyone buying new ones for a video rig, but with today's prices still pretty high I think that you should add the 2 2TB Greens in a RAID 0 configuration using two of the Intel SATA ports. Put your media and projects on the 1TB Samsung, completely ignore the older 300GB, and put media cache, media cach DB, all scratch files, and outputs (DVD etc) to the 2x2TB RAID array. You will need Win7 to use GPT since the array will be over 2TB before you format the RAID 0 array.

Also, if this setup is not fast enough for your media and your patience, my suggestions would be (in this order):

- add an nVidia GTX 570 video card and configure Premiere Pro 5 or 5.5 to use the Mercury Playback Engine (MPE) capabilities

- get a good cooler and overclock to 3.7GHZ

- increase RAM to 12GB or 24GB (RAM is very cheap for X58 motherboards now)

- think about adding more 7200rpm drives

I serously doubt you will need anything past the first suggestion, and you may even be satisfied without putting in a MPE video card.

Regards,

Jim

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Mar 15, 2012 Mar 15, 2012

Copy link to clipboard

Copied

Hi Jim,

Thanks for your reply and very helpful comments. I did not realise that the graphics card was a weak link - in my first post I called it a HD5700 althought the fuller model name is 'Powercolor ATI Radeon HD 5770 1024MB GDDR5' ... does this make any difference to your comments? If not then I will look at the GTX570.

I'm pretty sure I will be upgrading and your sequence looks great - especially as items 2 and 3 are not so expensive ... smile. I'm not sure if my current cooler qualifies as 'good' - it's a 'Arctic Cooling Freezer 7 Pro Rev 2 CPU Cooler (Socket 939/AM2/AM3/775/1155/1156/1366)'. I've never tried overclocking with it as no real need but any thoughts?

On the hard drives, I see your point about trying the 2x2TB drives and I've nothing to lose as they are here. If they test my patience then I can renew - I had not looked recently at hard disk prices and had a bit of a shock!!

Thanks again for your help.

regards

Claire

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Mar 15, 2012 Mar 15, 2012

Copy link to clipboard

Copied

Claire,

Your cooler is fine for some overclocking, your video card is probably a decent card for many applications, just not Adobe CS5/5.5.

For more information about MPE, check out:

http://forums.adobe.com/thread/773101

And for more basic information for overclocking:

http://forums.adobe.com/thread/529916?tstart=0

Finally, if you have other non hard drive questions, you should probably just start a new thread. This thread is already pretty long and focused on storage.

Regards,

Jim

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Mar 15, 2012 Mar 15, 2012

Copy link to clipboard

Copied

Hi Jim,

Thanks again for your help. Been reading loads and I'm sure there will be more questions - and understand about thread 'deflection'.

Regards

Claire

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Feb 23, 2012 Feb 23, 2012

Copy link to clipboard

Copied

#5 for life... worked very well for me...thanks!

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Mar 14, 2012 Mar 14, 2012

Copy link to clipboard

Copied

Are hard drives still recommended over ssd for scratch disks, or has that changed recently?  With the elevated prices of hard drives, I'm wondering if I should be looking at a ssd drive instead?  I'm thinking 500g in raid 0 ($400) vs a 240g sdd ($300)?

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Mar 14, 2012 Mar 14, 2012

Copy link to clipboard

Copied

First off, SSDs still aren't durable enough for long-term use for repeated writes and rewrites. You see, SSDs still have a limited number of rewrite cycles. At the rate the scratch disk is rewriting, you would have used up the SSD's entire life cycle in as little as just a few days! And the factory warranties do not cover such abusive wear and tear. Thus, in such a circumstance, you might have to pay as much as tens of thousands of dollars every week just to even replace those SSDs.

Second, NEVER RAID any SSD unless it's specifically designed for RAID! Raiding a consumer SSD will completely defeat the TRIM function and will not only degrade performance, but will also cause even more frequent failures. In the worst cases, a RAIDed SSD array will fail after only a few minutes of operation.

These are the reasons why hard drives are still better suited for those situations that require multiple repeated rewrites in such a short period of time despite the higher hard disk prices. But now, hard drive prices are now slowly beginning to fall: A month ago, the 1TB WD Black cost all of $250 plus shipping at Newegg. Now, it costs $140 plus shipping - still about $50 higher than the amount that same drive cost just six months ago.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Mar 19, 2012 Mar 19, 2012

Copy link to clipboard

Copied

Okay, after reflecting on the couple of replies which had fine advice and also reading many threads I've decided on a semi-upgrade - new GTX570 graphics, RAM to 12GB (only 3 slots on my mobo) and new 2 x HDD.

So, 3 questions

- any views on the Samsung F3 HD103SJ 1TB internal Hard Drive SATAII 32MB Cache 7200RPM ? These are ~ 78% of the cost of WD Caviar Black (£88 vs £112). Sometimes it's not made clear if a drive is 'green'. Many drives have phrases like energy efficient and so on and the Samsung site is useless for info. I'm thinking of getting two of these Samsung drives (unless advice otherwise).

- I already have a good 128GB SSD system drive and a 1TB Samsung HD103UJ. With the two new drives, I will follow the 4 disk option in the guidelines for disk usage chart. Does it matter which of the three uses (D or E or F) the older HD103UJ is used for or are these 3 drives so similar as to not matter ?

- I also have a spare Samsung 64GB MLC SSD. Is there any merit is installing this as well eg dedicated for the pagefile and/or other files handy to keep to one side.

Okay - a 4th question ... with the above disks are there any different configurations that make more sense ?

Just to clarify - my PP use is hobby-ist and so no delivery presures etc. Although nothing I do is critical, I do have a couple of 2TB 'green' drives that I will use for backups.

Many thanks,

Claire

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Mar 19, 2012 Mar 19, 2012

Copy link to clipboard

Copied

Claire,

I just went with the 500gb Spinpoint F3 HD502HJ in raid 0.  Which motherboard are you using?  One problem I had with an Asus p6t is that after 6 drives it uses an IDE controller and the performance decreased.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Mar 19, 2012 Mar 19, 2012

Copy link to clipboard

Copied

Hi and thanks for the replies.

Gf1317 - my mobo is Gigabyte GA-EX58-UD3R. I'm struggling to see the significant differences (capacity aside) between the F3 HD103SJ and the F3 HD502HJ that you mention. The Samsung site redirects to Seagate and the info is poor.

Bill Gehrke - the Seagate is around £100 with Amazon and so not too bad. I seem to remember reading that a certain generation of Seagate drives were to be avoided but the newer models are okay; is this correct.

Any thoughts on my other questions especially using my spare 64GB SSD for pagefile.

Many thanks.

Claire

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Mar 20, 2012 Mar 20, 2012

Copy link to clipboard

Copied

Claire,

Avoid Seagate's ending in .11 (11th gen. are "possessed"!).

Regarding your 64GB SSD, suggest you not use it at all. It simply will not bring much to the party for video editing. You already have a fast SSD for your OS/programs and the write speeds are so poor for SSDs that it will likely underperform your larger, newer rotating drives.

If I had a spare 64GB SSD I think I would want to find a high performance USB 3.0 case for it and use it like a USB stick - on steriods!

Regards,

Jim

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Mar 20, 2012 Mar 20, 2012

Copy link to clipboard

Copied

Jim,

What about using it for project files where you don't have much writing going on?  I've got two ssd in raid zero with 500mb/s read speeds.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guide ,
Mar 20, 2012 Mar 20, 2012

Copy link to clipboard

Copied

Gf,

You can certainly try it. I played with a 5xSSD RAID 0 array that benchmarks a 1.5GB (1500MB/sec) read speed and it only worked OK for project files. I simply love SSDs for OS/programs and laptops, but for some reason - likely sustained write speed - they just don't seem to excel with Adobe Premiere Pro CS5.

Jim

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Mar 19, 2012 Mar 19, 2012

Copy link to clipboard

Copied

Right now (today) the best performance and a great deal is the Seagate Barracuda ST2000DM001 at $130 for this 2 TB drive (or $65/TB)  Here are the fastest 7200 rpm transfer rates I have seen for HD Tune Wrire and Read rates for a two disk drive RAID 0 setup.

R0-2-x-2TB-Areca Write.jpg

R0-2-x-2TB-Areca-Read.jpg

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Mar 20, 2012 Mar 20, 2012

Copy link to clipboard

Copied

If the preview files and cache are on a seperate drive, what writting would be involved with the media drive? (other than occasional saves of your project file).  I understand the importance of having the cache on a non ssd drive, but I thought just having the media files on a ssd would be good?

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Mar 20, 2012 Mar 20, 2012

Copy link to clipboard

Copied

Jim & Gf,

Many thanks for the responses and ideas. I now understand about not using SSD's where there is very frequent writing. A problem with my limited knowledge is I'm not too sure which categories (ie media, projects, pagefile etc etc) of materials are accessed / written to so often as to make them non-starters for SSD.

So, you are saying that project files and media are suitable candidates for the 'spare' SSD - what about the pagefile ?

Thanks for you help.

regards,

Claire

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Mar 20, 2012 Mar 20, 2012

Copy link to clipboard

Copied

Claire,

Someone else will have to confirm that media files only get read from and your project file obviously only gets written to when you hit save.  At least thats my understanding and why I dumped them on a ssd drive.  Your pagefile, think of it as system memory that resides on a drive.  Like memory it gets read/written constantly and would be better served by a platter drive, in raid configuration for best performance.  With the drives I had laying around I settled on this for my system:

c: system -ssd

d: pagefile, cache and preview files - hdd raid 0

e: media files, project files- ssd raid 0

external nas in raid 5 with all my media files backed up.  I accept the hours I put into the project file being lost if I loose a raid 0 ssd drive.  I guess I could move that, but don't really know how a project file affects the performance of premiere.  If HD were not so expensive now, maybe I would look into a four drive system, per previous recommendations.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Mar 10, 2016 Mar 10, 2016

Copy link to clipboard

Copied

Hello,

How much sdd storage space is necessary for media cache files and previews files? I want a feedback, please.

at the moment I work with xavc Intra and long  1080*1920. In near future I will work with 4k xavc

The sequences is arround 1h.50 timeline duration

I thinking about

Kingston hyperx 3K  240GB for media cache files

Sandisk extreme pro 240GB for preview files and Export

what do you think ??? this space storage is enought??

all the best

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guru ,
Mar 11, 2016 Mar 11, 2016

Copy link to clipboard

Copied

LATEST

Ideally you want a 512GB SSD for the media cache and AE cache. A 256GB SSD will handle it but you will have to clear it more often and the smaller size will use the endurance rating quicker. I would suggest a Samsung 850 Pro either way.

Eric

ADK

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines