Generic Guideline for Disk Setup

[Harm_Millaard]

There have been many questions about how to set up your disks.

Where do I put my media cache files, where the page file, and what about my preview files?

All these allocations can be set in PR, so I made this overview to help you find some settings that may be beneficial. It is not a law to do it like this, it is a generic approach that would suit many users, but depending on source material, workflow and backup possibilities, it is not unthinkable you need to deviate from this approach in your individual case.

The reasoning behind this overview is that you want to distibute disk access across as many disks as possible and get the best performance.

Look for yourself:

I hope this helps to remove doubts you may have had about your setup or to find a setup that improves performance.

[Guest]

Thanks Harm for answering my question - I should have known.

[El_Plates]

Thank you for this Generic Guideline for Disk Set-up, Harm. It should prove invaluable when I'm building my PC. (for which I'm currently choosing components).

My budget allows for a 4 Disk configuration, plus an extra SSD (intel 520 series) for After Effects CS6's new cache features that assist RAM previews.

I have some questions about the "pagefile". My understanding of how the Pagefile works is limited, so I need to ask how does the Pagefile location become an issue for Premiere Pro? I noted that your guidelines consistently suggest to Not locate the Pagefile on the same Disk as the Media, Exports, or Project files.

Taking that into consideration, I must now ask : How large do Pagefiles tend to be?

My C: drive will be an SSD, and must have enough capacity to store the OS, Pagefile, CS6 Production Premium, CS5.5 Master collection (if it's necessary to place CS5.5 on the same drive as the CS6 upgrades), security software, plus a handful of minor apps.

Edit : would the Pagefile cause much wear and tear on the SSD, significantly lowering the drives lifespan? how frequently would it be read/written to?

[Guest]

I'm curious about this as well. I have 1 SSD (M4) for os/progs, and (2x) 2TB Barracuda xt's and (1x) 1tb Barracuda (6gbps) coming. I'm assuming I'll use the 1TB for pagefile and media cache.

Being new to Windows I'm not sure how big these pagefile's get. I'll have 32gb of RAM initially, hopefully 64 eventually. Will my system be using the pagefile much? Also how much write space does media cache usually need? I'm wondering, do we need to leave this disk exclusively for this stuff, or can we put other non-project related files alonside it or would this hurt performance?

Sorry about all my inquires, I'm new to windows and multi-disk editing setup so lots of questions flying at once!

[FITB_STUDIOS]

Generally speaking you should make your pagefile 1.5 times your ram. With that said, that was the formula back before ram was so cheap and 64x made it capable to have more than 4gb's. I still use it though. Go into your pagefile set up page and check the box so you can manage your own pagefile. Then set the max and minimum size for the pagefile to be the same amount and remove or set the pagefile on your operating system drive to be the absolute minimum. I think I turn mine off totally So for me I have 32 gb of ram, and created a partition on my SSD Cached Drive (Old Vertex 60gb ssd and an old sata2 500gb drive.) and made a 60gb pagefile on that partition. Probably overkill, but I have the space for it. The other partition with the remaining 400gb's is for my media cache, and scratch disks. I have an internal raid 10 for my media files and project files and a 3Tb sata 3 for my exports. Everything get's backed up to my external raid 3 device.

Hope that helps.

-Paul

[Guest]

So, as I understand it the pagefile takes over when RAM is full? But I've also been told that the system is not smart enough to know what is RAM and what is pagefile, and since the pagefile is much slower some people say it negatively affects performance, and those with lots of RAM should effectively disable the pagefile. Is there any truth to this?

[PaulieDC]

But I've also been told that the system is not smart enough to know what is RAM and what is pagefile

Huh? Who on earth said that? If the "system" wasn't "smart enough" there would't BE a pagefile. Please stop taking advice from whoever said that. The odd thing is, the system IS obviously smart enough to know what it is because it WILL use it when ram is gone, so the second part of your statement is true where you said "since the pagefile is much slower some people say it negatively affects performance". The only reason the pagefile is slow is because it's on a hard drive. Yes, you need ram to avoid hitting the pagefile. Don't get "extra" ram, get the max your motherboard holds. Ram is very cheap right now, but that changes every couple years, so fill 'er up. My 2009 EVGA X58 motherboard states that its 6 slots max out at 12GB of ram, because in 2009 the biggest sticks were 2GB a piece. When 4GB sticks finally got released and the price FINALLY came down to earth, I filled my mobo and the 24GB runs just fine. Point is, spend the money to get all the ram possible, and then make your pagefile 8GB for Min and Max, no more than that, it isn't necessary. And put the pagefile on a different drive, don't leave it on your C drive. But I highly suggest you don't disable it, you never know what Adobe updates may come out where the pagefile is explicitly used by Adobe software developers to persist metadata that doesn't need ram-level speed and therefore using the pagefile for that keeps it out of ram. I develop software, we do these kinds of things all the time because the expectation is that there will be a pagefile. Don't disable it. Adobe software guys are aggressive in increasing performance and they will use deeper level of Windows to get there. Look at the dependency Adobe uses on the Windows Registry... if you don;t run Adobe software as Administrator in Vista or Win7 there will be some settings that won't save because the Registry needs admin privileges which everyone had in XP. And if the Registry save fails, Premiere/Photoshop/AE don;t alert you, they just go on their merry way.

Keep your swap file (pagefile, same thing), get all the ram you can fit, and edit and be happy!

[Another_Photographer-qoO2PF]

PaulieDC wrote:

And put the pagefile on a different drive, don't leave it on your C drive.

Paulie, if C: Drive is an SSD is there a performance hit from leaving the pagefile on the C: drive?  Since much of the pagefile access is random, it seems it might be better leaving on a fast C: drive than a spinning 7200RPM HDD.

[Harm_Millaard]

Windows does not allow running without a pagefile. If there is no pagefile found, Windows creates a dynamic one on C: and there is nothing you can do to prevent that.

The following rule of thumb can apply:

RAM 64, pagefile 8 - 16 GB

RAM 32, pagefile 16 - 32 GB

RAM 16, pagefile 24 - 48 GB

Of course this depends on your workflow and multitasking habits, but with disk space so cheap it is better to reserve more space than coming up just short.

It is best not to put the pagefile on an SSD, if you can put it on a raid array of conventional disks, because current SSD's, and especially the ones with SF controllers show a large write degradation over time before they reach their stable state. Claimed write speeds of 450+ MB/s will soon deteriorate to around 250 MB/s or even less.

[UlfLaursen]

Harm, I once think I saw you recommend a NAS server brand, but I don't recall which one. I am going to use it purely for archiving and exchange between PC's. I have a Netgear NAS Pro now, but have been looking at a Synology. Do you have an oppinion on that?

Thanks.

/Ulf

[Harm_Millaard]

Synology is very good too. I have a Thecus and the 3-rd brand to look at would be QNAP.

Here is an overview of NAS systems to look at IMO:

http://tweakers.net/pricewatch/cat/685/nas-behuizingen.html#filter:ZY5LC8IwEIT_y5x7aPpQ2x9Q8CAIehMPN...

[UlfLaursen]

Thanks a lot Harm

Ulf

[El_Plates]

Thank you for info about the Pagefile. I definitely won't be locating it on the SSD now, and I'm considering not using an SSD for the C: drive at all. My current 7,200rpm drive might have to do for OS, Programs, Pagefile.

I still wish to know how, and when Pagefile can impact on Premiere Pro performance.

Harm's guideline for disc setup puts it on a seperate disc to Exports. That tells me the Pagefile is being either written to, or being read from during the Export rendering process. Is that so?

And what about when scrubbing though the Timeline?

When does the Pagefile come into play?

You'll all be sick of reading the word "Pagefile" by now lol

[Another_Photographer-qoO2PF]

El_Plates wrote:

When does the Pagefile come into play?

That's actually rocket science.  See here:  http://www.zdnet.com/blog/bott/windows-7-memory-usage-whats-the-best-way-to-measure/1786

If you can figure it out let us know.

All I've been able to gleen is that a pagefile is not like an old fashioned swap file.  It comes into play even if you haven't run out of physical RAM.  I *think* that Windows puts stuff there that it deems to be low priority.

Each page is 4kb, so there the write operations test the random not the sequential skills of the drive.

If you put it on a conventional hard drive, do so while the drive is nearly empty and has been defragmented, and don't let windows manage it.  Instead pick a number and set both the minimum and maximum size of the page file at that number.  That way, you're assured of one large contiguous unfragmented area for the file.

If you use and SDD it doesn't matter whether or not it's contiguous.

[Harm_Millaard]

One thing is sure, your exports are written to disk. If the pagefile were located on the same disk and it is used during the transcoding, you encounter a bottleneck, especially with limited memory. As I said in the original post, it is all about spreading the load of disk accesses across as many disks as you have. If you are in a situation where the pagefile is used as temp storage by the OS and you need to write the export file, putting them on the same disk does not adhere to the rule of spreading the load.

The pagefile is being used by the OS automatically when free memory is getting low. Now what is low? That is something that Windows decides, but if you use material like Red or Epic, that limit is reached much sooner than when you use DV material. If you use DL often and have AE open, then the limit is reached earlier than without AE and DL's. There is no clear cut answer for everybody. It depends on your workflow, source material, export settings and your hardware.

[Guest]

Should/Can anything else (non editing related files) go on the pagefile/media cache drive, or would this negatively affect performance? I'll have a 1TB drive for this drive personally.

[Another_Photographer-qoO2PF]

Harm Millaard wrote:

It is best not to put the pagefile on an SSD, if you can put it on a raid array of conventional disks, because current SSD's, and especially the ones with SF controllers show a large write degradation over time before they reach their stable state. Claimed write speeds of 450+ MB/s will soon deteriorate to around 250 MB/s or even less.

What if you can't put the page file on a RAID array of conventional disks?  At 250 MB/s it's still more than twice as fast as my fastest HDD.

[Harm_Millaard]

In that case, put it on the SSD.

[RFDPiper]

So for 24GB RAM, 24-36GB for pagefile?

[Harm_Millaard]

Yes.

[El_Plates]

I'd like to ask a few more questions about disc set up, plus ask for some suggestions for choice of disc model /type, and backup options. please.

I must preface this with a brief overview of my workflow: I use photoshop projects, and DSLR footage to create 1080p After effects compositions, which are then exported as Uncompressed .avi files upto 1491516kbps bitrate. The After effects exports are then edited in Premiere Pro before their final export in Youtube friendly formats.

Setup :

1/ is a Raid0 configuration relatively easy for a beginner to build?

2/ Do most Motherboards (including Z77's) support Raid cards, and how much should I expect to pay for a good one (if I need a card/hardware solution)?

3/ How many sets of Raid0 can I have per computer if there's one Raid Hardware/card?

Disk choice :

1/ what model/brand/spec'd drive would you recommend for Raid0 ?

2/ If I don't go Raid0, then what brand/model disks are best for my needs? (I was considering WD Velociraptors for discs D: E: F: G: because WD claimed that model has good write speeds).

Backup:

Once-a-day backing up is fine by me, so I ask : what is the best software for that, and can it be configured in a way that won't impact on system performance?

[Harm_Millaard]

1. Yes.

2. No, unless you accept a 10-15% performance penalty, caused by the video card. Prices from $ 400 to $ 1500.

3. Up to 14 per card, depending on the model.

1. Seagate Constellation ES, Hitachi 7K3000 Ultrastar, WD RE4.

2. Seagate BarracudaXT,  WD Caviar Black, Hitachi 7K3000 Deskstar.

[El_Plates]

Thank you Harm. You've helped me greatly in figuring out what components I need, and why.

I now understand why your what PC to build guide suggests vastly differing components lists for the "economical" and the "warrior builds. Around two thirds of the warrior budget goes into disk drives, and the components necessary to make the most of them.

If a Z77 motherboard doesn't offer proper support for a raid card, then I'd assume a Z68 wouldn't either.

Next I must figure out whether I want to spend $5,600 or only $2,500 on this build.

I may have to sell a kidney or two if I choose the Warrior option.

[Harm_Millaard]

The problem is that all 1155 platforms have only 16 PCIe lanes available. They are all used by the video card up to the moment you add a PCIe-8x raid controller (or a BM or Matrox card), because then the video card is throttled down to PCIe-8x, causing the performance loss.

If you opt for a X79 2011 platform, you have 40 PCIe lanes available and the video card continues to use PCIe-16x speed.

It is always the question how much are you willing to pay, because you get what you pay for.

See Adobe Forums: Planning / building a new system. Part 1

I thought the prices of second-hand kindeys were down very much, like the prices of second-hand houses, so that may not help you very much.

[skeeze]

Harm,

You had mentioned that you felt the Seagate SATA drives were faster than the Wester Digital Caviar Blacks.  From the raw performance metrics that were in this post by toms hardware, those numbers agree with you (continuous writes and reads are much faster)...however, as we all know benchmarks don't necessarily mean real world differences and even the posted benchmark #'s when it comes to loading pictures and video (albeit much less sophisticated apps), Seagate vs. Western Digital was about the same.  I just purchased several Caviar black drives(raid configuration) and I can still send them back, but I was just curious when you say the are faster, 1) did you benchmark them or was it based on more observational (which coming from you still counts!), and 2) is it a big enough difference you would recommend going through the pain of getting new drives and sending these back?

Thanks in advance.

[Harm_Millaard]

The reason the Seagates are faster than the WD Caviar Blacks is because they use higher density platters and that means less head movement and that translates in less time to position the heads, but both are very good disks to use for single operation and even in raid0. For parity arrays the Caviar Blacks are definitely out, because of the TLER problem. I have not yet heard of similar problems with the Barracuda 14 series, but that does not mean they are not there, only that I haven't heard of them.

I just purchased several Caviar black drives(raid configuration)

If the intended raid level is 0, 1 or 10, then these disks are good. If you are planning to use a parity array, raid 3 / 5 / 6, then I would exchange them for Seagates or Hitachis. In other cases I would not bother to get them exchanged. There are even pro's to holding on to the WD and that is the warranty period. Seagate gives only one year warranty, WD (IIRC) three years.

For my own build that I'm working on and document for others, I wrote this:

Ouch! So expensive

Since prices have not yet dropped to pre-flooding levels, this will be the most expensive component in the system.

With current price levels and assuming I will leave my current system mainly intact, this means I will need quite a lot of disks. At this moment I consider getting (at least in the first stage) the following setup:

Disk	Type	#	Configuration	Capacity	Purpose
C:	Corsair Performance Pro 256GB	1	Single disk*	256 GB	OS & programs
D:	Seagate Barracuda ST2000D	4	Raid0	8 TB	Pagefile, media cache, previews
E:	Seagate Barracuda ST2000D	16	Raid30, 2x(7xR3+ Hot-spare)	24 TB	Media & projects
F:	Seagate Barracuda ST2000D	5	Raid3*	8 TB	Exports

The Seagate Barracuda ST2000D is probably the fastest SATA disk available at this moment, has the best price per GB and has a low sound level. This is of course important with 25 of those disks in the system. All the files that can easily be recreated are on the raid0, so the risk is negligent in case of disk failure. The important raid, drive E: with the media and projects on it, has a hot-spare for each raid3, in addition to the parity disk. Of course I lose the effective storage space of 4 disks with such a setup, but it buys me safety in case of disk failure.

However, there is one complicating factor specifically for Europe and that is that legal warranty is two years, no matter what the manufacturer says. In this case Seagate gives only one year warranty, so the consequence is that every shop that sells these disks has to pay the 2-nd year of legally required warranty out of his own pocket. For many shops this simply means they no longer sell these Seagate disks. The risk is too big, especially since these disks have - again - a high failure rate, many are DOA or develop screeching noises. Period.

Luckily I'm not in a hurry, so I can look around for alternatives, but the choices are very limited. WD RE4 are still SATA 300 disks, WD Caviar Black are notoriously bad for parity raids, Hitachi 7K3000 Deskstars may be an option, of course the Hitachi Ultrastars are much better but carry a corresponding price tag.

You see you are not alone in having doubts about what is best, but there is one consolation, what is best today is outdated tomorrow, but still works.