Copy link to clipboard
Copied
How do you set up your editing machine in terms of disks for maximum performance and reliability? (SSD's are left out here.)
This is a question that often arises and all too often one sees that initial settings are really suboptimal. These rules are intended to help you decide how to setup your disks to get the best response times. Of course the only disks in an editing machine must be 7200 RPM types or faster. No GREEN disks at all.
Rule 1: NEVER partition a disk. You may ask why? First of all, it does not increase disk space, it just allocates the space differently. However, the major drawback is that for a partitioned disk the OS must first access a partition table at the beginning of the disk for all accesses to the disk, thus requiring the heads to move to the beginning of the disk, then when it has gotten the partition info move to the designated area on the disk and perform the requested action. This means much more wear-and-tear on the mechanics of the disk, slower speeds and more overhead for the OS, all reducing efficiency.
Rule 2: Avoid using USB drives, since they are the slowest on the market. Do not be tricked by the alleged bandwidth of USB 2.0 advertisements, because is just is not true and remember that the alleged bandwidth is shared by all USB devices, so if you have a USB mouse, keyboard, printer, card reader or whatever, they all share the bandwidth. Stick to SCSI or SATA disks or e-SATA. If needed, you can use Firewire-800 or even Firewire-400 disks, but they are really more suited for backups than for editing.
Rule 3: Use at least 3 different physical disks on an editing machine, one for OS/programs, one for media and one for pagefile/scratch/renders. Even on a notebook with one internal drive it is easy to accomplish this by using a dual e-SATA to Express card connector. That gives you an additional two e-SATA connections for external disks.
Rule 4: Spread disk access across as many disks as you have. If you have OS & programs on disk C:, set your pagefile on another disk. Also set your pagefile to a fixed size, preferably somewhere around 1.5 times your physical memory.
Rule 5: Turn off index search and compression. Both will cause severe performance hits if you leave them on.
Rule 6: If the fill rate on any of your SATA disks goes over 60-70% it is time to get a larger or an additional disk.
Rule 7: Perform regular defrags on all of your disks. For instance, you can schedule this daily during your lunch break.
Rule 8: Keep your disks cool by using adequate airflow by means of additional fans if needed. You can use SMART to monitor disk temperatures, which should be under 35 degrees C at all times and normally hover around 20-24 C, at least in a properly cooled system.
Rule 9: If people want raid, the cheapest way is to use the on-board IHCR or Marvell chip, but it places a relatively high burden on the CPU. The best way is a hardware controller card, preferably based on the IOP348 chip. Areca ARC and ADAPTEC come to mind. 3Ware uses it's own chipset and though not bad, they are not in the same league as the other two. Promise and the like in the budget range are no good and a complete waste of money. Expect to spend around $ 800 plus for a good controller with 12 connectors internally and 4 e-SATA connectors. Important to consider in a purchasing decision is whether the on-board cache memory can be expanded from the regular 256/512 MB to 2 or even 4 GB. Be aware that 2 GB cache can be relatively cheap, but the 4 GB version extremely costly ($ 30 versus $ 300). For safety reasons it is advisable to include a battery backup module (BBM).
Rule 10: If you can easily replace the data in case of disk failure (like rendered files), go ahead and use raid0, but if you want any protection against data loss, use raid 3/5/6/10/30/50. For further protection you can use hot spares, diminishing downtime and performance degradation.
In general when you get a new disk, pay close attention to any rattling noise, do perform regular disk checks, and in case of doubt about reliability, exchange the disk under guarantee. Often a new disk will fail in the first three months. If they survive that period, most of the disks will survive for the next couple of years. If you use a lot of internal disks like I do (17), set staggered spin-up to around 1 second to lessen the burden on the PSU and improve stability.
Hope this helps to answer some basic questions. If not, let me know. Further enhancements and suggestions are welcome.
Copy link to clipboard
Copied
Harm Millaard wrote:
How do you set up your editing machine in terms of disks for maximum performance and reliability? (SSD's are left out here.)
This is a question that often arises and all too often one sees that initial settings are really suboptimal. These rules are intended to help you decide how to setup your disks to get the best response times. Of course the only disks in an editing machine must be 7200 RPM types or faster. No GREEN disks at all.
What do you mean by "GREEN disks"?
Copy link to clipboard
Copied
Green disk are in marketing terms ECO friendly disks that use less energy, because their rotation speed is lower. Like a car, if you do not go over 40 MPH you use less fuel than when driving 70 MPH. Slowing down the slowest component in your computer is bad, like a snail hitting the brakes before cornering.
Copy link to clipboard
Copied
Thanks for the explanation Harm. Seems like those "Green" drives would be best suited for backup type functions.
Copy link to clipboard
Copied
Rule 10: If you can easily replace the data in case of disk failure (like rendered files), go ahead and use raid0, but if you want any protection against data loss, use raid 3/5/6/10/30/50. For further protection you can use hot spares, diminishing downtime and performance degradation.
Is there a reference link you could provide as to what all the different levels of RAID do?
Copy link to clipboard
Copied
I actually changed a few 'green' discs to black caviar because of your advice, Harm, and I have been really satisfied. Thanks.
/Ulf
Copy link to clipboard
Copied
can you please eleborate on the difference you experienced?
Copy link to clipboard
Copied
Hi Shaluda,
Here you could find some good information about the different RAID-levels (click on the graphics to make the demo play):
http://www.raid.com/04_01_00.html
Nicolaj
(and thanx Harm for you advice "a couple of posts ago"...)
Copy link to clipboard
Copied
Thanks for the link Nicolaj; much appreciate it!!
Copy link to clipboard
Copied
Is there a reference link you could provide as to what all the different levels of RAID do?
THIS will give you the specifics of the levels of RAID.
Good luck,
Hunt
Copy link to clipboard
Copied
This can also be informative:
Copy link to clipboard
Copied
removed
Message was edited by: Frederic Segard
Copy link to clipboard
Copied
Thanks for the interesting information. Two questions:
Is a seperate scratch disk for basic editing (AVCHD, little video effects) really needed? Can't you just use a disk for OS/programs and a storage disk for media, project file and preview files?
Scratch disk setup in premiere offers options for captured video and video previews. Assuming captured video means raw source video, this will point to the storage disk. So a seperate scratch disk would only be used for video previews, am I correct?
The WD green 1TB does go to 7200 when needed and therefore seems as fast at other storage disks. Is green really that bad?
@Ulf Larsson: what difference did you notice by going from green to black?
Copy link to clipboard
Copied
Is a seperate scratch disk for basic editing (AVCHD, little video effects) really needed? Can't you just use a disk for OS/programs and a storage disk for media, project file and preview files?
It is about the same as having a car with a manual gearbox. Sure you can use that car and drive wherever you like while using only the first and second gear. However, if you also use 3-rd, 4-th and 5-th gear, it will take less time and entail less risks in traffic. Same with the number of disks.
So a seperate scratch disk would only be used for video previews, am I correct?
No, it will be used for the media cache. the indexed, conformed and peak files and possibly for the pagefile, as well as all preview files.
Mechanical disks are the slowest components in a computer. To get optimal results you need to get the last bit of performance from them. Even if what you say is true that the WD green disk can adjust their speed from 5400 to 7200, consider the enormous amount of time needed to get up to speed. That is an eternity in computer terms. Two or three seconds to stabilize at 7200 is extremely slow when disk access is measured in ms. And why? There is no discernable price difference or it can be measured in peanuts.
Copy link to clipboard
Copied
thanks for your reply. What confuses me is that premiere does not offer scratch settings for these categories you mentioned: media cache. the indexed, conformed and peak file. Only captured video and preview video.
Assuming a 3 disk setup (OS, scratch, storage), and setting captured video to storage disk (raw footage) and preview video to scratch disk, which disk is used by premiere for what?
Copy link to clipboard
Copied
The media cache location can be set in the preferences and contains the indexed, conformed and peak files.
Copy link to clipboard
Copied
Hi
I did not have any exact problems with the green ones, I just wanted to be safe based on Harms knowledge. I use the green ones for other purposes like backup and stuff.
/Ulf
Copy link to clipboard
Copied
you say you can set media cache in preferences, but I looked and it only mentions captured audio/video and preview audio/video. but I think I understand what you mean: when you set captured video to scratch disk in those preferences, media cache is one of the temporary files that will be stored on the scratch disk, correct?
so for example, I have this setup:
disk 1: OS etc
disk 2: scratch
disk 3: storage with raw footage
I start a new project, I set captured audio/video and preview audio/video to scratch disk 2. And I save the project file itself to disk 3.
After I import video into the project from disk 3, and start editing, all scratch files, including media cache, confirmed audio, preview files, etc will now be stored on scratch disk 2. Is this how it works?
Copy link to clipboard
Copied
Correct.
Copy link to clipboard
Copied
thanks for pointing out this media cache setting. But I am confused again.
I opened a project on my current PC (just 2 disks, one OS and one data). Scratch disk settings under project/project settings/scratch disks is ''same as project'' for captured video/preview
Than I looked under edit/preferences/media and the location for media cach files is a folder on my C (OS) drive: user/.../AppData/etc.
Than I tried to locate that media cache folder on my PC to see what's in it. There is no AppData/Roaming/Adobe/etc folder on my PC
Than I looked in the project folder on my data disk and next to project file it has these folders:
auto-save
preview files
encoded files
media cache files
this makes my wonder: how did I get a media cache file in my project folder on data disk while the setting in premiere points to my OS disk? And where is that folder mentioned under media cache in premiere? And why didn't they just add media cache to scratch disk settings?
Copy link to clipboard
Copied
You ticked the box in the upper right part of my screenshot.
Copy link to clipboard
Copied
no, it's unticked (I never touched it). I am starting to believe this edit/preferences/media is simply not working. The media cache setting is overruled by the scratch disk settings.
Copy link to clipboard
Copied
Hello Harm,
if you look at scratch disk settings in premiere, what is the ''captured video/audio''' setting actually used for? There is no folder on my harddisk called captured video, while there are folders called media cache and preview files made by Adobe. Maybe captured video = media cache, overruling the media cache setting in preferences, explaining why I do have a media cache folder within my project folder but not on the path mentioned under media cache setting?
Copy link to clipboard
Copied
.... i was just wondering.. all these talk about HDD performance.... why did you leave out SSD again? as per my other posts i am planning to build the BEST machine money can buy (whilst still being a somewhat practical) and SSD was my first choice dedicated for OS Drive and Scratch/temp files.. was just wondering why you left it out.. is it a bad idea?
Copy link to clipboard
Copied
At the moment I wrote this article, SSD's were too expensive and did not offer a significant speed/performance gain over a good raid setup.
One can still wonder at the current time (october 2009) whether they will offer any significant performance gain with the new controllers and trim function in Win7 over old fashioned hard disks.
For OS, SSD's are approaching the level where you can consider them, a single 160 GB Intel SSD for around € 520 here in comparison to a single 150 GB Velociraptor for around € 140.
For editing the capacity is still a major bottleneck. I prefer a 12 TB raid30 with 800+ MB/s transfer rate for around € 1600 (including a top notch raid controller) over a 640 GB SSD raid0 with around 750 MB/s transfer rate for around € 2080.
Copy link to clipboard
Copied
Harm,
First of all, I'd like to congratulate you on your excellent articles about your hardware configurations. I've just ordered a new pc and I'd have a question for you. I would like to set up a system with one HDD for my OS and 4 HHD's in raid 0 config for my main storage with a hardware raid card. But mostly on the net when you reed about raid, it's always a "total" raid config (with all the HDD in raid & OS also on the raid) and mosstly a software raid.
Also some scary story's about suddenly having a pc that's not bootable anymore...
So my question is how about am I to go to work ? I tought of the following scenario :
1) set up pc with 1 HDD and install win7 ultimate 64bit on it.
2) open up pc, put hardware raid controller card & 4 HDD in it. Also connect the 4 HDD to the hardware raid card.
then I'm not so sure, do I have to go into the BIOS at next startup to set up the 4 HDD in raid (or is this only needed for software raid ?)
Or do I only have to config the 4 HDD with the setup software that comes with the raid card ?
Sincerely,
Frank from Belgium