Disk Cache & Separate Drives

[kevinh23223536]

"For improved performance, choose a disk cache folder on a fast hard drive or SSD separate from your footage, and allocate as much space as possible."

Hi,

I'm currently running a RAID 0 with 2, Samsung 970 PRO M.2 drives (full desc. below). Now I'm a bit worried that my workstation is not optimal for After Effects, considering all my video files, my OS + AE application + Disk Cache are all going to this one drive (since RAID 0 combines both of my M.2 into one drive making it a lot faster, but less stable). Also running 64GB DDR4 RAM @ 3400 MHz.

(SAMSUNG 970 PRO M.2 2280 512GB PCIe Gen3. X4, NVMe 1.3 64L V-NAND 2-bit MLC Internal Solid State Drive (SSD) MZ-V7P512BW)

Here's a comment by 2 different individuals regarding this:

The rules are different with SSD drives versus mechanical HDD's. SSD drives have a latency in microseconds where as mechanical drives have a latency in milliseconds. This means SSD drives can read and write far more files in a given time segment than mechanical drives. If you have an SSD then cache and media files can go there without issue. You can also include cache files easily with an OS drive on an SSD. Mechanical drives are where the separate drive philosophy really developed from. Although you can run media and cache from the same drive provided the drives and interface ie e-sata, USB3 or TB are fast enough. Firewire and USb2 would not be

Eric-ADK

Tech Manager

You don't want the same drive/bus being used for varying read and write operations at the same time. That is a useful principle. But having cache files on the same drive as the OS and applications is in alignment with that principle; the OS is mostly read from the disk at boot, the applications are mostly read from the disk at application start, and the cache files are read and written during program run time. There is very little overlap. What you do _not_ want to do is have cache files on the same disk/bus as footage files or exported files, since those assets are being used at the same time.

Todd Kopriva, Adobe Systems Incorporated 

After Effects quality engineering

This comment however was back in 2013. I'm currently running a high end M.2 SSD that reads at 3550MB/s and writes at 3450MB/s which is incredibly fast.

How much performance am I losing by having a setup where I'm running everyone on 1, RAID 0 drive using 2 M.2 SSD's? I also have a NAS HDD in order to store/archive all of my completed videos, etcetcra.

Thank you.

[Peru Bob]

Moved to the Hardware forum.

[StevRo]

Hi Kevin,

A lot has changed since CS6 (which this was written about) in terms of storage the advice is still sound.  SSDs are a good uplift over platter drives.  For OS/app drives NVME drives won't give you much of an uplift over a sata 6 SSD.  Where NVME can see a uplift if source footage is >200MB/sec or it is used as a scratch drive as writes are ~15 to 20% faster than a conventional sata 6 SSD. output/archival can be done with a platter drive as it is the least resource intensive and most cost effective. Hope it helps.

[kevinh23223536]

Hi StevRo,

Thanks for the reply. In dummy-terms, could you simplify what the conclusion to my question is? lol

[StevRo · Accepted solution]

Hi Kevin,

How much performance are you losing?:  None,  Windows/AE can't use the all the read/write speed of one 960 pro, much less two 960 pro's in raid 0.

Backstory:  At the time the quotes were made, consumer SSD were relatively expensive/GB (13-14x compared to platter drives).  I believe in late 2013 consumer SSD prices ranged in price from 70c-80c/GB where drive capacity topped out at 512GB.  To get more capacity or speed you had look at other solutions such as dedicated raid controller, thunderbolt or PCI/e SSD which cost a lot more $/GB.   At the time few posting on the AE forum were well heeled enough to afford $7,000+ workstation, so adobe decided to focus on the larger consumer market rather than a high end niche market.  It is why you see the recommendations use the highest performance device per dollar in the places where it mattered most.

[kevinh23223536]

Thank you ^^

Was getting slightly worried about my new setup hehe

[tristansummers]

I think you will be getting significantly faster read times on the RAID 0 because the latency goes down. Even modern nvme drives do not match the 19 μs latency of the IOFX and I had 4 in a RAID which is the only scratch drive I have had that consistently plays dpx without caching. I thought the real time playback was just part of the updates in AE until I tried just using two iofx, or usiing one Corsair mp510. Microseconds, and 5 parallel writes really help. NOt sure how many parallel writes you get with two 960 Pro.  I am now deciding between 2 x iofx and 2x 10870Ti or one GPU and 4 IOFX as the render times reduce with the write speeds. NOticeably.

I tested it. because I can write 20 things at once instead of 1. Now if only we could get multi processor working again... renderrrgardennnnn