Known Participant

Answered

Hardware question regarding export

Forum|Forum|2 years ago
July 20, 2023
3 replies
736 views

Hey everyone so I'm exporting a single 5min scene in 4K BRAW to pro res 4444xq. I am on Nvdia accelerated...im writing to a fast 7tb SSD but my Read drive in a 40+ TB Raid 0 for max speed. My stats are showing my GPU and CPU are barely being used. The render takes 30min...for the hardware I have this seems slow but I could be wrong can anyone make any suggestions. Thank you hardware below

Processor: AMD Ryzen 9 7900X 12-core, 24 thread Processor

*Motherboard: GIGABYTE X670S AORUS Elite AX (AM5/LGA 1718/Ryzen 5000/AMD/X670/ATX/DDR5/Quad M.2/PCIe 5.0/USB 3.2 Gen2x2 Type-C/AMD WIFI 6E/2.5 GBE LAN/ Q-FLASH Plus/M.2 EZ- Latch)

KONYEAD Quad PCIe NVMe M.2 SSD Adapter Card-PCI Express 3.0 x8 Card Support 4X M.2 NGFF PCIe NVMe SSD (ASM2824 Switch)-Support Non-Bifurcation Motherboard (3002K)

Blackmagic DeckLink 8K Pro with Quad Link 12G-SDI

*Memory: T-Create Expert Overclocking 10L DDR5 64GB 6000MHz (PC5- 48000)

*GPU (Graphics Card): NVIDIA PNY GeForce RTX 4090 OC XLR8 EPIC-X RGB Triple Fan Gaming Verto 24GB

*Power Supply: 1000 Watt - CORSAIR RM1000X Power Supply - 80 PLUS Gold, Full Modular

*Processor Cooling: CORSAIR Hydro Series H100i PRO 240mm RGB Liquid CPU Cooling System - [Ryzen]

*Boot Drive: 500 GB Samsung 860 EVO SSD -- Read: 540MB/s, Write: 520MB/s - Single Drive

*Primary Hard Drive (Write Drive/ Render/Cache Drive): Samsung 980 Pro SSD 2TB PCIe NVMe Gen 4 Gaming M.2 Internal
Solid State Memory Card (x4) (8TB total) (RAID 0)

*Primary Hard Drive (Read Drive/ Intake Drive): Seagate Exos X16 12TB 7200 RPM 512e/4Kn SATA 6Gb/s 256 Mb Cache (x4) (43.6TB Total) (RAID 0) USB 3.0 enclosed in a
Mediasonic 8 Bay harddrive tower.

*Storage Hard Drive: 4 TB Seagate Barracuda PRO Hard Drive -- 128MB Cache, 7200RPM, 6.0Gb/s - Single Drive

*Sound Card: 3D Premium Surround Sound Onboard

*Speaker System: Creative A250 2.1 Speakers System - Largest-in-class, down-firing subwoofer; Dual Slot Enclosure design

*Network Card: Onboard LAN Network (Gb or 10/100)

*Operating System: Windows 11 PRO

*Case Fans: Noctua NF-A12x25 (120mm) (x5)

*Keyboard: Adobe Premiere Pro Astra Logic Keyboard

*Panels: Tangent Wave 1 Color Grade Panel

*Monitor (x2): Flanders Scientific DM241 Monitor

Dell Ultra Sharp PremierColor UP3017

*External Calibrator: X-Rite i1 Display Pro (x2) (Standard and Flanders Proprietary)

*Software: Adobe Creative Suite (Neat Video, Pluraleyes)

Davinci Resolve Studio (Neat Video, Film Breath & Gate Weave, False Color)

Topaz AI

*Headphones: Audio-Technica ATH-M50x Professional Monitor Headphones

MISC: An extensive VFX, SFX, and 3D asset library

This topic has been closed for replies.

Correct answer RjL190365

It's not the hard drives per se, in your case. It's the USB 3.0 connection to your external RAID array. You see, USB 3.0 still cannot sustain more than about 400-ish MB/s total per channel. And having all of those HDDs sharing a single USB 3.0 channel is complete murder on performance. You'll never get more than about 400 MB/s in total throughput no matter how many drives are in that RAID enclosure. That's slower than even a single SATA SSD, let alone an m.2 PCIe SSD. (And please don't confuse MB/s with Mbps or Gbps; they are completely different measurement units. USB 3.0's maximum throughput is only 450 MB/s sustained, or 500 MB/s peak, based on the 5 Gbps theoretical maximum throughput rating, 8 bits per byte, and the use of 8b/10b encoding.)

In other words, if you're going to use an external RAID array, you should get a Thunderbolt 3 or Thunderbolt 4 RAID box and a Thunderbolt 3 or Thunderbolt 4 controller adapter. Unfortunately, no AMD CPU-powered platform natively supports Thunderbolt of any version, and the few Thunderbolt solutions which are compatible with AMD systems tend to be problematic. Only Intel CPU-powered platforms have a properly functioning Thunderbolt interface.

R

RjL190365Correct answer

Legend

It's not the hard drives per se, in your case. It's the USB 3.0 connection to your external RAID array. You see, USB 3.0 still cannot sustain more than about 400-ish MB/s total per channel. And having all of those HDDs sharing a single USB 3.0 channel is complete murder on performance. You'll never get more than about 400 MB/s in total throughput no matter how many drives are in that RAID enclosure. That's slower than even a single SATA SSD, let alone an m.2 PCIe SSD. (And please don't confuse MB/s with Mbps or Gbps; they are completely different measurement units. USB 3.0's maximum throughput is only 450 MB/s sustained, or 500 MB/s peak, based on the 5 Gbps theoretical maximum throughput rating, 8 bits per byte, and the use of 8b/10b encoding.)

In other words, if you're going to use an external RAID array, you should get a Thunderbolt 3 or Thunderbolt 4 RAID box and a Thunderbolt 3 or Thunderbolt 4 controller adapter. Unfortunately, no AMD CPU-powered platform natively supports Thunderbolt of any version, and the few Thunderbolt solutions which are compatible with AMD systems tend to be problematic. Only Intel CPU-powered platforms have a properly functioning Thunderbolt interface.

D

defaultoox0p3kd078aAuthor

Known Participant

Thanks for this, let me run this by my teams texh support guy to see see if he has any suggestions as well per your advice to see what can be done per my set up. Thank you for taking the time to present a detailed explanation

Peru Bob

Community Expert

@RjL190365

D

defaultoox0p3kd078aAuthor

Known Participant

And what I mean by feasible is cost. I mean for around 40TB of ssds that's in the upwards of 10grand just for intake storage. I also just figured Braw isn't a super heavy format I mean we have a dedicated tech guy with our team who's done tons of bench marking. Using Puget bench marking too...im not saying your wrong but it seems just like an amazing cost for a file format that to me isn't super heavy lifting compared to arri or RED

Peru Bob

Community Expert

I've moved this to the Video Hardware forum.

Peru Bob

Community Expert

*Primary Hard Drive (Read Drive/ Intake Drive): Seagate Exos X16 12TB 7200 RPM 512e/4Kn SATA 6Gb/s 256 Mb Cache (x4) (43.6TB Total) (RAID 0) USB 3.0 enclosed in a
Mediasonic 8 Bay harddrive tower.

*Storage Hard Drive: 4 TB Seagate Barracuda PRO Hard Drive -- 128MB Cache, 7200RPM, 6.0Gb/s - Single Drive

By @defaultoox0p3kd078a

HDDs are a weak link. They should be fast SSDs.

D

defaultoox0p3kd078aAuthor

Known Participant

I appreciate your response. I guess where I'm confused is 40TB of ssds seems unrealistic to me. The storage needs at baseline for intake is 40TB for a feature if shooting something on red or arri alexa depending on the format one is looking at upwards of 100TB of storage for intake. I just don't see how that's feasible with SSDs

Warren Heaton

Community Expert

If you've been purchasing storage to the past two to three decades, 32TB or 64TB of SSD storage isn't that expensive.

I paid just under $3,000 for an external Micropolis 3GB SCSI-II drive back in 1994. At the time, it was a must have strorage for mobile full-frame, full-motion 480i60 Radius VideoVision compressed QuickTime for broadcast quality.

OtherWorld Computing is a vendor that I like for high-capacity, external SSD storage:

Something I really like about their Thunderbay line is that we can mix and match high speed SSD with high capaicty standard drives to balance overall performance and capacity.

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded