40
Replies
You're totally ignoring the basics of the media you're using. That's where you're going sideways.
That is all very long-GOP media, which rags the crud out of the CPU/RAM/cores/threads subsystems. So adding in a ton of RAM is actually the best thing you could do. The cameras and their highly specialized chips that create/write that media record on a complete "I" frame every 15 to 30 ... or for some drones ... 120 frames apart. In-between the record a matrix-based dataset of the pixels that have changed since the last I-frame, will change before the next, or ... both. Now there are even partial I-frames. Which is how they extend it up to 120 frames for complete I-frames.
The computer has to create the I-frames & store them, pull up data-sets, recall the I-frames & calculate the in-between p & b frames. Store those as well as pass them onto the video system, create the next frame for the video. On & on. I know colorists with massive rigs who don't even think once about working 6k RED & Arri media. But as soon as they see long-GOP stuff, that gets transcoded to replace the original media on the sequence they're going to work.
Here's a bit from the GoPro folks from the manual for their own editing app:
So ... yea, the RAM's good to get. And you'll probably still want to make proxies if you do much color, Warp Stabilizer, video de-noising, anything like that.
Neil
Neil, actually I am not ignoring the media at all. I have studied I frames, All-I, compression, bitrates, resolution, DI, etc, etc. I know more about how consumer/prosumer grade cameras record, store, and compress their recorded footage than I ever thought possible. I am fully aware of how this type of media completely trashes CPU, RAM, and GPU; my problem is my system does not SHOW me that any of that is going on. The only place I seem to run low sometimes is RAM which is why I am throwing more of that at the problem.
I have no problem throwing more hardware at the problem, but I need the system performance tools to just tell me what I need more of. I can run 1080P 8bit at 30FPS from these same cameras in real time all day long within AP; warp stabilizer, transitions, adjustment layers, you name it it's buttery smooth; so I feel like I'm close to being able to do the same thing in 4K, I just need to figure out what the weak link is.
I have also not overclocked anything, I have been very careful to not install any additional codecs, this whole workstation was built from the ground up just for video editing. Below you can see what I mean. Task Manager shows that absolutely nothing in the HW is being taxed, CPU, RAM, GPU, and HD IOPS are not even 50%, yet the video at this exact point in time is dropping about 20 frames at a time.
redmodel: I checked the priority of the PP executable and it is set to normal.
Bill, I just ran your benchmarking tool, below is a screen shot of the results. I did take into consideration that RAID 5 is not the best for performance, but this is not ordinary RAID 5, this is an NVME storage spaces array using Windows 10, so the slight performance hit I took on the stripes I knew the NVMI RW throughput would easily be able to overcome for video editing. 
Bill, I'm not sure your test is accurate. The test was not run from the Storage Spaces array, it was ran from the C:\ drive which is a single dedicated NVME drive. According to PassMark my NVME drive is performing within it's expected range, so I'm not sure if your test scripts are accurately measuring the disk performance as the following screen shot shows. I am running Premier Pro CC 2018, not sure if that makes a difference.
I think I am just stuck with CPUs that cannot keep up like ECBowen said. I probably should have done a custom build with an Intel i9 using a custom rackmount chassis. My only hope is that the inbound RAM will make a difference.
PiterSpb, I cannot use QSV because my CPUs do not support it. I did try both enabling and disabling H.264 acceleration and it did not seem to make much difference.
I was asking both in the camera maker's booths (Samsung, Panasonic, and DJI) and the software for Avid, PrPro, and Resolve at NAB about the long-GOP media. The camera spokes-persons all say they have a custom setup chip on or with some customized firmware setups for their specific needs. Meaning even between cameras within one brand, their media can be encoded differently.
So any hardware-based solution needs to be able to work with rather a wide range of encoding possibilities of the various custom made chips along with customized firmware and processor options in the various cameras.
The software folks all said long-GOP is the grungiest ... crud ... they get stuck trying to sort out. Avid guy wasn't particularly interested in even spending much time on it. "Transcode, my friend ... in whatever app might make something actually editable out of that ... (long pause) ... media." I got the distinct impression he really didn't want to have his internal thoughts quoted. 
Resolve people were more interested in talking, but still ... noted that Resolve on most computers will need to be set to use Optimized Media (Resolve's t-code process) for any decent playback for editing, color, or sound.
Adobe engineers were actually more willing to spend time talking about it. But clearly, my impression is that they feel like they're chasing a phantom target. There's something like over a hundred different cameras they are trying to accommodate at any one time. And each seems to come out with new firmware or models every few months that change the encoding settings used. In ever more creative ways. So it's in reality maybe a couple hundred or more encoding settings they're needing to match up at any one moment.
Which ... again ... is why in the aisle-talk, several conversations were about suggesting to Adobe's video managers that MediaEncoder be fitted to 'read' as many camera/device media forms as possible to make t-codes from many of them, and PrPro perhaps be built around using a few major editing codecs to get both better playback and stability while editing. Which might make more sense in general. And just give up on trying to keep up with the amazingly creative folks storing camera media on cards.
Intel's QuickSync on some of their CPU's is supposed to be faster at handling H.264 than the standard circuits of the CPU, but I don't have any information on how well that works. So there is a start at least on the hardware side. Would sure be nice if there was a lot more capability in that hardware though.
Neil
AdChoices