Generic Guideline for Disk Setup

[Harm_Millaard]

There have been many questions about how to set up your disks.

Where do I put my media cache files, where the page file, and what about my preview files?

All these allocations can be set in PR, so I made this overview to help you find some settings that may be beneficial. It is not a law to do it like this, it is a generic approach that would suit many users, but depending on source material, workflow and backup possibilities, it is not unthinkable you need to deviate from this approach in your individual case.

The reasoning behind this overview is that you want to distibute disk access across as many disks as possible and get the best performance.

Look for yourself:

I hope this helps to remove doubts you may have had about your setup or to find a setup that improves performance.

[Harm_Millaard]

Randall and Scott,

Thank you for your accurate and factual observations.

[Personal comments removed]

Message was edited by: Jerry Klaimon

[Another Photographer]

Harm Millaard wrote:

Randall and Scott,

Thank you for your accurate and factual observations. They are a blessing after reading some ill founded fairy tales.

Harm,

I purchased a Crucial M4 SSD (the one with appro 240GB).  I will be installing Windows 8 and CS6 on it soon.  I have a 3-disk setup, per your chart at the beginning of this.  The other 2 disks are Caviar black 2TB and Seagate 3TB 7200rpm.

Given all we've learned, where should the pagefile go??

[Guest]

Modern computers with tons of RAM pretty much don't do a paging operation ever because they almost never approach the end of useable RAM space.  I'd put it on either of those secondary drives because Windows does technically do something minor to the pagefile as it initially boots (like checking if it's still there and the correct size) but after that, it doesn't affect system performance other than taking up an equivilant amount of space as your system memory unless you manually set it lower and tell it to automatically expand if needed.

Those drives are so close in performance that which one to use as a cache file drive would come down to whicher has better benchmarks if you can find them.  I think the seagate might have a slight edge in speed though, if I'm guessing which exact models each are properly.

If your SSD contains an individual folder of all the source files for whatever 1 project you're working on, Premiere will load those at 500MB/s because of the awesome read speed.  Then when cache files are generated from those files, writing them back to the same drive they're potentially being read from isn't a great idea so either one of those spinning drives should work fine.

Since they're so close though, logic would suggest using the 3TB one for storage of video clips and end results themselves and the smaller 2TB drive for cache files since you should have more clips and end results than cache in most cases.

[ECBowen]

VHC-CO-IT wrote:

Modern computers with tons of RAM pretty much don't do a paging operation ever because they almost never approach the end of useable RAM space.  I'd put it on either of those secondary drives because Windows does technically do something minor to the pagefile as it initially boots (like checking if it's still there and the correct size) but after that, it doesn't affect system performance other than taking up an equivilant amount of space as your system memory unless you manually set it lower and tell it to automatically expand if needed.

Be aware that atleast 1 application programmer/designer has verified on the forums what I and others have stated all along that many applications are still written to write directly to the page file regardless of the amount of ram available. The minimum page files size still needs to remain a decent size which seems to be around 16GB.

Eric

ADK

[JCschild]

i like the Crucials.. my first choice would be Intel but seriously the read speeds on both are very close write speed will not matter as an OS drive.. and the crucial is much less $..

[Harm_Millaard]

Bob, there are a number of things to consider to answer that, and the overriding one is the amount of memory installed. The more memory, the less the use of the page-file and the less impact it has where you locate it. A seond consideration is that a SSD is great for reading, but far less so for writing and the page-file is used for writing and reading if memory is tight. Add to that, that the 'stable state' performance degradation is certainly not bad on the M4, but still worse than on some other SSD's and writing will shorten the life expecancy of a SSD. An inherent shortcoming of flash memory.

What are your options? Given the relatively slow writes of the SSD and the shortening of its life expectancy, I would not put the page-file on the SSD. There is already a lot of writing going on, on this SSD by Windows (events, logs, FAT maintenance and pointers, user account, system tasks, etc.), so that leaves one of the conventional HDD's. But, assuming the 3 TB Seagate is a Barracuda XT from the 14 series, it is noticeably faster than the WD Caviar Black. Then the next thing to consider is the size of your media, as it influences the fill rate of your HDD and when the fill rate goes up over 60%, the performance goes down. Now the page-file will be limited in size, say 32 GB or less, so my initial reaction is to use the Seagate for that. It will not impact the fill-rate, it is on the fastest drive and especially if it is the first (static) file on the disk and will never be fragmented.

Hope this helps.

[Guest]

Harm Millaard wrote:

Given the relatively slow writes of the SSD and the shortening of its life expectancy, I would not put the page-file on the SSD.

I wouldn't recommend the page file on an SSD either but certainly not for that reason.  These outdated 2011 charts of writing non-compress-friendly data show just how much faster they are.  Plus, it's assumed that anything written to a hard drive is later read back at some point (otherwise why was it written?) so the 4-5x faster read speed comes into play.  Check it out:

http://www.tomshardware.com/charts/hdd-charts-2012/-04-Write-Throughput-Average-h2benchw-3.16,2904.h...

http://www.tomshardware.com/charts/ssd-charts-2011/AS-SSD-Sequential-Write,2783.html

Modern 2012 drives are quite a bit faster and can write 400MB/s easily, even with video files.  The newest Crucial M4's are practically specifically designed for video files since they make no attempt to compress anything ever and they're rated at about 250MB/s and are quite cheap.  Yeah, you'll nuke the drive within a couple years but a better SSD will be out at that point and a spinning drive and a 128GB SSD are almost exactly the same price.

As far as what ECBowen said, I can't imagine what significant amount of data would bypass the RAM to go straight into the page file.  Considering Premiere can choose from available memory or the cache folder to dump temporary data, it would have no reason to dump large project data files into the page file when there's enough RAM available.  Windows itself doesn't even consider touching it until about 75% RAM usage so as long as you put enough in, you can just about ignore the page file.

[Another Photographer]

Harm Millaard wrote:

Bob, there are a number of things to consider to answer that, and the overriding one is the amount of memory installed. The more memory, the less the use of the page-file and the less impact it has where you locate it. A seond consideration is that a SSD is great for reading, but far less so for writing and the page-file is used for writing and reading if memory is tight. Add to that, that the 'stable state' performance degradation is certainly not bad on the M4, but still worse than on some other SSD's and writing will shorten the life expecancy of a SSD. An inherent shortcoming of flash memory.

What are your options? Given the relatively slow writes of the SSD and the shortening of its life expectancy, I would not put the page-file on the SSD. There is already a lot of writing going on, on this SSD by Windows (events, logs, FAT maintenance and pointers, user account, system tasks, etc.), so that leaves one of the conventional HDD's. But, assuming the 3 TB Seagate is a Barracuda XT from the 14 series, it is noticeably faster than the WD Caviar Black. Then the next thing to consider is the size of your media, as it influences the fill rate of your HDD and when the fill rate goes up over 60%, the performance goes down. Now the page-file will be limited in size, say 32 GB or less, so my initial reaction is to use the Seagate for that. It will not impact the fill-rate, it is on the fastest drive and especially if it is the first (static) file on the disk and will never be fragmented.

Hope this helps.

Thank you Harm.  That is very helpful.  One follow up question.  Should I give any consideration to whether having the pagefile on a conventional drive will slow down RANDOM access to that drive?  Let's say the drive has to write 8 KB page to the pagefile.  The head would have to stop what's doing (for example reading a media file) and travel to where the pagefile resides (since it is contiguous).  So whereas the pagefile is not used much, is it slowing down the drive?

[Guest]

Scott Chichelli wrote:

Sanforce drives have had numerous issues since day one. (particularly OCZ)

Except OCZ doesn't use Sandforce controllers in their latest series or the one before it.  They use one made by Indillinx, you know, seeing as how they bought the company Indillinx.  I looked and found about 3 drives currently for sale that use the original glitchy sandforce controller.  The other 297 ones use flawless modern ones.  That outlines pretty clearly how accurate the rest of that post is too.

Speaking of that, an SSD will help video editing performance IMMENSELY!!!  Why wouldn't it? Seriously, answer that question.  Why would it not help from a technical standpoint?

Before you answer, here's...well, the answer:

It's really, really, really simple once you understand how computers work and how video editing works.  Let me explain. Video clips reside on a hard drive.  Unless you have some magical computer that can cache every single clip in memory all at once, those clips have to be read off the hard drive before they're processed by the CPU and in the system memory.  Even just playing them off the timeline means you're loading either the original or a cached file off the hard drive it resides on in order to play it or process it.

Here's the problem with that: If your computer can processor a video stream at over 120MB/s (and most can) but your spinning hard drive can only serve it up at 120MB/s, guess what you get to wait for.  All your fancy chip and immense memory and amazing CUDA cores all get put on hold because the data is still loading off the hard drive into the memory.

That's why Pr has something called a cache drive in the first place!!!  If disk speed didn't matter, put your cache files on the same drive as everything else.  They obviously want dedicated disk I/O for project files because the speed at which they're put on and read off the hard drive is that important. So get an SSD, seeing as how they're typically 4x faster than single spinning drive and 1/4 the price of a RAID array.

[JCschild]

Hilarious…

How do I know ssd wont help performance that easy I have benchmarks to prove it how about you? That’s what I thought.. oh please show me your benchmarks where performance is improved…

Scott

ADK

[Guest]

Okay:

The speed at which spinning drives can have their data read off in a large, sequential fashion like for example video files from the drive and into memory:

http://www.tomshardware.com/charts/hdd-charts-2012/-01-Read-Throughput-Average-h2benchw-3.16,2901.ht...

The speed at which SSDs can have their data read off in a large, sequential fashion like for example video files from the drive and into memory:

http://www.tomshardware.com/charts/ssd-charts-2011/AS-SSD-Sequential-Read,2782.html

I assume you weren't actually being serious since I think you know those numebrs as well as I do but for everyone else's reference, there's the side by side, real work comparison numbers.

[JCschild]

must be benchmarks with adobe..

[JCschild]

same test

I7 2600K 4.7 GHz

16GB Blackline 1600 CL 9

570GTX

4 WD 1Tb Sata 64 Meg Cache 600 Drives in 2 Raid 0 arrays

3 Layer - 30:46

4 Layer - 33:36

I7 2600K 4.7 GHz Z68

16GB Blackline 1600 CL 9

570GTX

4 WD 1Tb Sata 64 Meg Cache 600 Drives Raid 0 and 80GB SSD Cache Drive

3 Layer - 31:16

4 Layer - 35:08

I7 2600K 4.7 GHz Z68

16GB Blackline 1600 CL 9

570GTX

2 WD 1Tb Sata 64 Meg Cache 600 Drives Raid 0 and 80GB SSD Cache Drive

Quick Sync

3 Layer - 13:36

4 Layer - 17:20

I7 2600K 4.7 GHz Z68

16GB Blackline 1600 CL 9

570GTX

2 WD 1Tb Sata 64 Meg Cache 600 Drives Raid 1 and 80GB SSD Cache Drive

Quick Sync

3 Layer - 13:51

4 Layer - 17:50

[Guest]

Hmmm...let's see...lol.  I think maybe the fact that you ran a video rendering test when we're actually talking about how quickly hard drives can load video clips into Premiere is part of the problem, lol.  May I perhaps suggest a better test?  Import a 10GB video file into Premiere from a spinning drive and see how long it takes then do it from an SSD and see how long it takes.  The SSD will be 4x faster.

The more specific reason why your test didn't make any sense for an SSD as a cache drive is that the filter you were running resulted in an output stream of data slower than the write speed of a spinning drive.  In other words, run a simpler, easier filter that can be processed at several hundred MB/s output stream speed and you'll see an immense different with an SSD as a cache drive.

[JCschild]

That test is about as average you can get to a normal workflow and why its used.

i can play with synthetic tests all day long they are pointless and of no use.

import? the import is ONLY as fast as the drive being imported from

[This post has been edited to remove unacceptable personal comments]

Message was edited by: Jerry Klaimon

[JCschild]

a few benchmarks  note the ssd as os and caching was actually a few seconds slower than a standard sata.. also note the overclocked performance you know that thig you think should not be done.. lol 90% of posters on here have OCed systems..

Premiere Pro CS5 Testing

980X at 3.33GHz

12GB Redline at 1600 CL 6

Intel SAS 600 Controller

4 WD 1Tb Sata 64 Meg Cache 600 Drives in 2 Raid 0 arrays

Video material - AVCHD 1080P 24 Frame Each Cut to 30 minutes of material

Export Codec - H264 HDTV 1080P 24 Preset Default

4 Effects per Layer - Fast Color Corrector, Brightness & Contrast, Video Limiter, Sharpen

Each Layer Scaled to 50% for 4 frame PinP view.

285 GTX

3 Layers - 43:39

4 Layers - 52:46 plus 48 minutes effects render on 4th Layer

Quadro CX

3 Layers - 42:50

4 Layers - 47:34

980X at 4.0GHz

Quadro CX

3 Layers - 36:08

4 Layers - 45:22

980X at 4.0GHz

285 GTX

3 Layers - 36:13

4 Layers -  49:01 Plus 42 minutes effects render on 4th Layer

980X at 4.0GHz

285 GTX

4 Drive Raid 0

3 Layers - 35:10

4 Layers - 47:03

980X at 3.33GHz

285 GTX

4 Drive Raid 0

3 Layers - 41:47

4 Layers - 55:02

4 Layers Mercury Engine Software only - 2:10:00

980X at 3.33GHz

285 GTX

4 Drive Raid 0

1 WD 1TB Sata 64 Meg Cache 600 Drive for Media Cache and Temp Files

3 Layer - 41:38

4 Layer - 55:25

980X at 3.33GHz

285 GTX

4 Drive Raid 0

Intel SSD 160GB OS/Cache Drive

4 Layer - 55:27

[Bill Gehrke]

In my post above where I gave details on my SSD benchmarking failure those 2 TB drives are the 7200.14 (ST2000M001)  I now have 5 of those drives and 4 0f the 500 GB version.  They all are working well and show no signs of any problems and they are excellent performers.

[Harm_Millaard]

RAID5 is the way to go.  2 or more data drives, 1 parity drive.

Again incorrect. Raid5 is distributed parity. Try to learn the basics, before you make wildly incorrect statements.

I suggest you heed your own advise:

Okay, you need to go back to basic RAID training to see why that's nonsense.

[Media747]

Wondering if anyone could help me with what would be the better setup. I am going to have an msata drive as my boot, and then three 7200RPM hard drives.

So I was just wondering if I should replace one of my 7200 RPM drives with an SSD, and how that would help performance-wise? (And what I should place on that SSD). Should I be using the SSD as my boot and then the msata as another drive ? (And in case you're wondering, this is going to be in a laptop which is why there's an msata drive).

I also have an option to use my external / esata dock for another 7200 RPM drive if necessary.

[Guest]

To answer media 747's question, no spinning hard drive is going to even be on the same planet as a good SSD for your main booting drive.  I have a demo system with a little 60GB Intel maplecrest 330 SSD and when it's loading windows 7, the little floaty glowing ball things don't even touch.  They're supposed to form into a flag and then pulse while the OS loads but it finishes so fast, it doesn't even get that far.  It also loads the desktop, all applications, the antivirus, etc in 3 seconds.

My very high performance Seagate 7200 1TB drive in my home computer will do about 4 pulses on the flag and take at least 30 seconds to fully boot everything.  There's just no contest.  The only problem is, SSDs above 120-128GB are quite expensive!  Also, the majority of SSDs brands have quality control problems, inferior flash chips, bad wear leveling, etc.  It changes weekly but just look online to see which are the best.

[Another Photographer]

VHC-CO-IT wrote:

To answer media 747's question, no spinning hard drive is going to even be on the same planet as a good SSD for your main booting drive.  I have a demo system with a little 60GB Intel maplecrest 330 SSD and when it's loading windows 7, the little floaty glowing ball things don't even touch.  They're supposed to form into a flag and then pulse while the OS loads but it finishes so fast, it doesn't even get that far.  It also loads the desktop, all applications, the antivirus, etc in 3 seconds.

My very high performance Seagate 7200 1TB drive in my home computer will do about 4 pulses on the flag and take at least 30 seconds to fully boot everything.  There's just no contest.  The only problem is, SSDs above 120-128GB are quite expensive!  Also, the majority of SSDs brands have quality control problems, inferior flash chips, bad wear leveling, etc.  It changes weekly but just look online to see which are the best.

SSD boot drives are expensive on a per GB basis, but as a boot drive it is quite cheap.  Since a boot drive doesn't need to be above 250GB (120GB is sufficient, really), there is no reason NOT to get an SSD.  I splurged on a quality 250GB SSD, and it was about the price as a 7200rpm 2TB drive so "expensive" it was not.

On the other hand, don't expect it to make a huge difference to your life.  Yeah, it boots faster but with modern operating systems, there is not a huge reason to reboot all that often.  When I boot in the morning, I really don't care whether it takes 40 seconds or 20 seconds.  If you really care about that 20 second difference, just consider that you will spend a whole lot more time researching the "right" SSD drive, troubleshooting it, downloading firmware, tweaking and posting here about it if it doesn't work as you expected.

Right now, I am running Windows 8 from a VHD on the 7200rpm drive.  (That means I am basically not using the SSD at all except occasionally when I need to boot into Windows 7 for something.)  And you know what?  I don't miss the speed of the SSD for boot-up and starting programs.  When Windows 8 is released, I will install it on the SSD, but it's not such a big deal.

As far as quality changing weekly, I suppose that is the case if you are scraping the bottom of the barrel.  If you go for quality name brands like Intel, Samsung and Crucial, their quality does not change by the day.

My advice would be not to fret too much about benchmarks and "performance" when shopping for an SSD boot drive and instead go for quality and reliability.  TRIM or no TRIM, the performance will deteriorate and the drive will be upgraded sooner or later, and cheaper, more reliable SSDa are just around the corner, so don't spend too much and don't get anything bigger than you need for "future proofing".

[KJerryK]

Empty post

[KJerryK]

This thread has been going on for two years without any problems, and then in the last few days we've received a number of abuse notifications. I've been reading them and see that three or four posters are more and more making personal comments against those with whom they disagree. This is a technical forum and there will inevitably be differences of opinions. But using comments such as "that is foolish" or "you don't know what you are talking about" or even "you are mistaken" has no place here. They are inflamatory phrases and contribute nothing. Please make your case in technical terms, Or if you can't just do not post.

You'll notice I posted this in reply to myself so I don't appear to single out any one individual. You know who you are.

[TerriStone]

Well said, Jerry.

As the forum guidelines say:

"Personal insults, verbal attacks, and generally disrespectful, offensive, or abusive messages will be edited or deleted by forum moderators. Repeated violations will result in temporary suspension of forum access, eventually leading to being banned from forums. "

Those of you using inflammatory terms such as Jerry describes above, be warned that you are very close to temporary suspension.

[dan_public2]

I'll reply to Jerry because I agree 100%.  And I don't exclude myself from this advice because some of my remarks lately could be considered uhmm...  "intemperate".   So my suggestion below is meant for all, including me.

I believe that the key issue is not understanding how our words impact others.  On the Web, we are not who we think we are. We are what we write.  Therefore, it's is our responsibility to ensure that our opinion gets across clearly while still showing respect to others. 

My suggestion...

NEVER, EVER use "you" or "your" except when speaking positively about someone else.   For example, if you write "Your idea sucks!", you're attacking the other person's idea. Or even worse is "You suck!" (or some variation).  That's attacking the core person. 

You may believe you're just strenuously stating your opinion, but virtually all readers will see you attacking someone else.  Very bad.   Worse, the target WILL react negatively even if he/she agrees with part of what you've written!   What's really bad is that most people will never read beyond the harsh words!   You have a fight and NOT a discussion.

Instead, make it impersonal.   For example,  write "OK, but I have different facts." or "Interesting point, but I disagree."   Notice that now the focus on the idea and NOT the person. Now it's just a discussion of ideas and facts where everyone is free to weigh in. 

In any case, remember that the other person may be right or partially right in the context of their needs and constraints.   If you respect that other person first, you may find that you're both right.  It's just that you have different needs and constraints. Or you may find that you're wrong and you've just learned valuable information.   Everyone walks away with dignity intact and a little more knowledgable.

I can honestly say that every person in this and other contentious threads has added value to the discussion. We just need to show respect to others and then present our own viewpoint.

All of this is IMO.

Regards,

Dan.