I'm in an ongoing quest to make my Windows machine as performant as possible, especially with Lightroom and Premiere.
My computer isn't anything too special:
Windows 10
Intel 6600K processor
32 GB Ram
2TB NVME storage for Windows, apps and Catalog
6TB 7200 RPM HDD for Photos and Video files
One aspect that I've always wanted to address is the bulk storage for my photos/videos. Ideally I would buy a large NVME (either PCIE or M.2 based) but that gets expensive really fast.
I recently came across "Storage Spaces" which is primarily used in the Windows Server space, and in Windows 10 it is primarily used to do Windows based RAID for disks. Hidden away in PowerShell though, is the ability to "attach" very fast SSD or NVME to relatively slow HDDs to help with the performance.
I decided to pick up a second NVME storage (in this case a Samsung 970 EVO PLUS 1TB) and try it out.
Here is what it looks like in Windows once it is setup:
Here is a diagram that describes it:
My 6TB drive which normally reads at 150MB/s and writes at 130MB/s now has a couple of caches in front of it:
- Write Back Cache. This is used for when writing photos/videos to the drive. In my case I've configured it with 1/10th of the size of my NVME drive - 100MB. Now when I write photos/videos to my storage drive, it writes them at 3200MB/s! Each night (its weekly by default) Windows will run optimization at 1am and offload the write back cache - ready for the next job.
- Read Cache. The remainder of my 1TB NVME is configured as a Read Cache. Windows maintains a "Heat Map" that marks which files are accessed the most - these are mirrored into the Read Cache everytime the Optimizer runs. This means that often accessed photos/videos will be read at 3500MB/s instead of at 150MB/s.
Storage Spaces is well documented:
https://docs.microsoft.com/en-us/powershell/module/storage/?view=win10-ps
Unfortunately, as I mentioned above, the commands to set this up are hidden away behind PowerShell commands. I found this guide:
https://nils.schimmelmann.us/post/153541254987/intel-smart-response-technology-vs-windows-10
WARNING:
The following is not for the faint of heart. If you are not comfortable with disk partitioning, formatting drives then I'd recommend getting help before proceeding. It is possible using the commands below to erase the contents of your drives! You have been warned!
Instructions:
Note when I wrote this up I was using a spare 500GB SSD and a 3TB HDD which is what you will see below. Secondly I suggest you read Nil's page (linked above) and become familiar with it as well as all of these instrunctions before proceeding with real hardware.
Pre-requisites:
# You must be in Administrator mode in Powershell
# Have backups of your photos, videos and anything else you value
# Your SSD/NVME and HDD that you plan to use in the Storage Pool must all be empty. You can't just migrate your photo drive with all its photos/videos into a storage pool.
#The lines that are bolded below are meant to be copy pasted into your powershell window (you did rememeber to run this as Administrator, right?)
# Variables to be loaded for the Storage commands that follow. Copy and paste these into Powershell. Note, you can validate if it worked with the echo command (for example: echo $StoragePoolName)
# Note you can change the Names below to suit your preferences. Don't change the ResiliencySetting unless you know what you're doing and have multiple disks.
$StoragePoolName = "My Storage Pool"
$TieredSpaceName = "My Tiered Space"
$ResiliencySetting = "Simple"
$SSDTierName = "SSDTier"
$HDDTierName = "HDDTier"
# List all disks that can be pooled and output in table format (format-table)
Get-PhysicalDisk -CanPool $True | ft FriendlyName,OperationalStatus,Size,MediaType
# My output looks like this. If a disk is missing, then it is not in the correct uninitialized state - see diskpart commands at the bottom of this post.
FriendlyName OperationalStatus Size MediaType ------------ ----------------- ---- --------- WDC WD30EZRX-00DC0B0 OK 3000592982016 Unspecified Samsung SSD 850 PRO 512GB OK 512110190592 SSD |
# Do not proceed if you don't see the drives that you want to Pool together listed!
# If the MediaType is set to either SSD or HDD then you don't need to run this.
# However, if MediaType is "Unspecified" from the above output for either an SSD or HDD then it must be updated as follows
Set-PhysicalDisk -FriendlyName "WDC WD30EZRX-00DC0B0" -MediaType "HDD"
# Store all physical disks that can be pooled into a variable
$PhysicalDisks = (Get-PhysicalDisk -CanPool $True | Where MediaType -NE UnSpecified)
# Create a new Storage Pool using the disks in variable $PhysicalDisks with a name of My Storage Pool (or whatever you renamed it to)
$SubSysName = (Get-StorageSubSystem).FriendlyName
New-StoragePool -PhysicalDisks $PhysicalDisks -StorageSubSystemFriendlyName $SubSysName -FriendlyName $StoragePoolName
# View the disks in the Storage Pool just created
Get-StoragePool -FriendlyName $StoragePoolName | Get-PhysicalDisk | Select FriendlyName, MediaType
# The pool should show all the disks you plan to use (both SSD and HDD)
FriendlyName MediaType ------------ --------- WDC WD30EZRX-00DC0B0 HDD Samsung SSD 850 PRO 512GB SSD |
# Create two tiers in the Storage Pool created. One for SSD disks and one for HDD disks
$SSDTier = New-StorageTier -StoragePoolFriendlyName $StoragePoolName -FriendlyName $SSDTierName -MediaType SSD
$HDDTier = New-StorageTier -StoragePoolFriendlyName $StoragePoolName -FriendlyName $HDDTierName -MediaType HDD
# Identify tier sizes within this storage pool
# NOTE: In my experience these values don't work in the creation of the virtual disk below
$SSDTierSizes = (Get-StorageTierSupportedSize -FriendlyName $SSDTierName -ResiliencySettingName $ResiliencySetting)
$HDDTierSizes = (Get-StorageTierSupportedSize -FriendlyName $HDDTierName -ResiliencySettingName $ResiliencySetting)
# This is the size of the Write Back Cache. In this example it is 100GB. Adjust this to the size you want and copy/paste this into your powershell
$WriteBackCacheSize = 100000000000
New-VirtualDisk -StoragePoolFriendlyName $StoragePoolName -FriendlyName $TieredSpaceName -StorageTiers @($SSDTier,$HDDTier) -StorageTierSizes @(($SSDTierSizes.TierSizeMax - $WriteBackCacheSize - $SSDTierSizes.TierSizeDivisor * 8), ($HDDTierSizes.TierSizeMax - $HDDTierSizes.TierSizeDivisor * 8)) -ResiliencySettingName $ResiliencySetting -AutoNumberOfColumns -WriteCacheSize $WriteBackCacheSize
# All done, now you can use Disk Management and create a volume and format it, and assign a drive letter and copy your photos/videos over.
DISKPART Instructions - Only if needed!
# If the drive is not found with to be poolable in the Get-PhysicalDisk command above then use diskpart to remove the partitions and "clean" it:
# You don't need to run these commands if your drives were previously unuse.
# Use diskpart to select and clean the disk:
diskpart
list disk
#Select the disk that needs to be fixed for example disk 3
disk 3
clean
exit
# You may need to reboot (I did) for disks to be found before proceeding
AdChoices