• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
0

Premier Pro - Stutters on 4K GH5 and 4K DJI Video

Community Beginner ,
May 07, 2018 May 07, 2018

Copy link to clipboard

Copied

I cannot figure out why my video editing workstation still stutters on GH5 4K 8bit content and DJI P4Pro 4K H.264 content. I have read every Adobe Premier Pro performance tuning guide I can get my hands on and I still cannot smoothly playback and edit 4K content from these two sources.I keep throwing more hardware at the problem and it's not improving. Below is my hardware platform:

Dell 7910 Rackmount Workstation

2 x Xeon 6 core processors

16GB DDR4 memory

NVIDIA GTX 980ti Video Card

RAID 5 NVME PCIe RAID array

NVME PCIe card for the OS

For my workflow I always color grade and add transitions last yet I still have to create proxy files sometimes when I need to view some of the clips in realtime because the stuttering is completely unworkable. I just ordered 80GB more DDR4 memory but I doubt it will help anything.

The problem is, I have no problems throwing more hardware at the problem if I could just figure out where the problem is. I watch GPU, CPU, disk, and RAM performance when it is stuttering and everything is below 5% utilized. The one thing I have seen is memory utilization sometimes hits 80% which is why I am upgrading the memory. I don't understand how the software can stutter when nothing is being heavily utilized. The CPU usually hovers around 5%, the GPU around 2-7%, the disk less than 1% and memory usually around 30-60%. This is with a clean install of Windows 10 64bit, and with playback set to 1/8 quality. I have seen users online get better results with way less hardware.

Yes I know I could create proxy files, and yes sometimes I do create proxy files because I don't have a choice, but with my hardware specs I feel I should not need proxy files. All of my content is 4K H.264 and mostly GH5 8bit footage or DJI Phantom P4 footage stored in an MOV container. Output renders are fast but editing is completely frustrating.

[Moderator note: moved to best forum for technical help.]

Views

8.6K

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
May 07, 2018 May 07, 2018

Copy link to clipboard

Copied

You're totally ignoring the basics of the media you're using. That's where you're going sideways.

That is all very long-GOP media, which rags the crud out of the CPU/RAM/cores/threads subsystems. So adding in a ton of RAM is actually the best thing you could do. The cameras and their highly specialized chips that create/write that media record on a complete "I" frame every 15 to 30 ... or for some drones ... 120 frames apart. In-between the record a matrix-based dataset of the pixels that have changed since the last I-frame, will change before the next, or ... both. Now there are even partial I-frames. Which is how they extend it up to 120 frames for complete I-frames.

The computer has to create the I-frames & store them, pull up data-sets, recall the I-frames & calculate the in-between p & b frames. Store those as well as pass them onto the video system, create the next frame for the video. On & on. I know colorists with massive rigs who don't even think once about working 6k RED & Arri media. But as soon as they see long-GOP stuff, that gets transcoded to replace the original media on the sequence they're going to work.

Here's a bit from the GoPro folks from the manual for their own editing app:

So ... yea, the RAM's good to get. And you'll probably still want to make proxies if you do much color, Warp Stabilizer, video de-noising, anything like that.

Neil

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
May 07, 2018 May 07, 2018

Copy link to clipboard

Copied

Neil, actually I am not ignoring the media at all. I have studied I frames, All-I, compression, bitrates, resolution, DI, etc, etc. I know more about how consumer/prosumer grade cameras record, store, and compress their recorded footage than I ever thought possible. I am fully aware of how this type of media completely trashes CPU, RAM, and GPU; my problem is my system does not SHOW me that any of that is going on. The only place I seem to run low sometimes is RAM which is why I am throwing more of that at the problem.

I have no problem throwing more hardware at the problem, but I need the system performance tools to just tell me what I need more of. I can run 1080P 8bit at 30FPS from these same cameras in real time all day long within AP; warp stabilizer, transitions, adjustment layers, you name it it's buttery smooth; so I feel like I'm close to being able to do the same thing in 4K, I just need to figure out what the weak link is.

I have also not overclocked anything, I have been very careful to not install any additional codecs, this whole workstation was built from the ground up just for video editing. Below you can see what I mean. Task Manager shows that absolutely nothing in the HW is being taxed, CPU, RAM, GPU, and HD IOPS are not even 50%, yet the video at this exact point in time is dropping about 20 frames at a time.

Stuttering-Now.JPG

redmodel: I checked the priority of the PP executable and it is set to normal.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
May 07, 2018 May 07, 2018

Copy link to clipboard

Copied

I've had this stuff explained to me by tech wizards. Felt incredibly mentally enlightened for a few minutes, but the extra gray cells didn't seem to stay "full". Ah well.

It has to do with the processing subsystems interactions ... not just the CPU or RAM total usage figures, but how well the data moves in/out/through the CPU, caches, RAM, and the order things get run through various threads. That much I did retain. Oh, and how the software uses the various resources it sees. I've read of those able to work through a system and figure out where the hangup is. But I don't know all that, and wish I could help more.

I do know there's another user who suggested a minimum of 10-12GB of RAM per core for H.264 processing in a different thread.

Neil

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
May 07, 2018 May 07, 2018

Copy link to clipboard

Copied

Oh ... probably a very good thing is to get actual hard data on that machine working within PrPro ... Bill Gehrke's PPBM8 testing package is pretty quick to run, and you get hard data comparing your rigs various systems under stress to thousands of other hardware tests. Actually very useful. As is Bill's advice ... Bill Gehrke​

Tweaker's Page ... http://ppbm8.com/index.html

Neil

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
May 07, 2018 May 07, 2018

Copy link to clipboard

Copied

I ran Novabench to check my system and it gave my GPU a pretty terrible score (14%). I think it is because my video card is in the PCIe x8 slot instead of in a PCIe x16 riser which means it's only got half the bandwidth it can use. I knew this would be a problem during the system build but I did not want to wait for the riser to come in. Now it would take a complete rebuild to install the riser and move the video card to the x16 riser since the Windows OS is installed to an NVME RAM drive in the other slot.

I still do not think it is the GPU though since the GPU never goes over 10%. I think it's more likely that the RAM will make a difference.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
May 07, 2018 May 07, 2018

Copy link to clipboard

Copied

Thank you Neil

Dell 7910 Rackmount Workstation

Have you tuned that Dell to get rid of all the garbage that they install at the factory?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
May 08, 2018 May 08, 2018

Copy link to clipboard

Copied

Yes this is a clean install, I started with blank hard drives and built it from there. Not a single Dell utility is installed on this workstation.  I've also tuned it for performance in the BIOS, I've dedicated 13GB of memory to Premier Pro within its settings, I have verified that NVIDIA CUDA is set for playback rendering, I have dedicated an NVME drive for the cache media, and I have the absolute latest firmware and drivers for every hardware component. I also tested creating a new sequence from the source media - nothing helps. Anything from the GH5 or P4Pro at 4K resolution stutters badly. I've got RAM on the way, if that does not fix it then I may order the x16 riser and rebuild the whole thing again.

The chassis is so crowded though the VC may be throttled due to temperature problems but I still do not think that is it since the GPU never goes over 10%.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
May 08, 2018 May 08, 2018

Copy link to clipboard

Copied

Sounds like a good setup, how about testing with my Premiere Pro BenchMark (PPBM)?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
May 09, 2018 May 09, 2018

Copy link to clipboard

Copied

Now that I reread this thread there is one item that jumpedout for me, RAID 5 is lousy for any function in video editing.  It is designed for data base applications where your have lots of short read/write activity, not sequential files.   When I get a few minutes I have a GH5 10-bit file sample that I am going to play with later 

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
May 09, 2018 May 09, 2018

Copy link to clipboard

Copied

Bill, I just ran your benchmarking tool, below is a screen shot of the results. I did take into consideration that RAID 5 is not the best for performance, but this is not ordinary RAID 5, this is an NVME storage spaces array using Windows 10, so the slight performance hit I took on the stripes I knew the NVMI RW throughput would easily be able to overcome for video editing. PPBM-Results.JPG

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
May 09, 2018 May 09, 2018

Copy link to clipboard

Copied

Bill, hopefully you can tell me if the numbers are good or bad, I checked the results against the ones on your website but did not see a way to match up the numbers in the results to the ones I got.I bumped up my RAM order to 128GB, so I should know by this weekend if RAM is holding me back.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
May 09, 2018 May 09, 2018

Copy link to clipboard

Copied

This is insane.

Your test results

"1.  Your Disk I/O time = 115 seconds.

This means your Premiere disk write rate is 322.54 MB/second."

Yet you make this statement

"I did take into consideration that RAID 5 is not the best for performance, but this is not ordinary RAID 5, this is an NVME storage spaces array using Windows 10"

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guru ,
May 09, 2018 May 09, 2018

Copy link to clipboard

Copied

The issue has to do with those CPU's and how inefficient and complex H264 4k is. Because H264 4K is so compressed and complex, you need much faster GHz cores more than extra threads to process the media in the time period required to maintain realtime playback. Normally as a general rule you want a CPU that is 2.6GHz or higher for standard media to maintain playback. 4K media can and will raise that specifically with codecs like H264 or H265 for example since the compression ratio is high and complex. This means the CPU's have to do far more work in the same period of time to generate the frame data for playback. Those Xeon 2620 V3's processing latency is just to high for that. Even the new Intel i7 8700K 6 Core can struggle at times maintaining H264 4K playback in an editor and those run at 4.0 to 4.7Ghz. For that system you really need to convert those files to DNxHR and then that system likely will handle the playback fine. Cineform would work as well. However H264 4K just will not run well on that system and doesn't run well on any Dual Xeons.

Eric

ADK

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
May 09, 2018 May 09, 2018

Copy link to clipboard

Copied

Thanks, definitely not what I wanted to hear though. I wanted a 2U rackmount workstation and the Dell was the only one that seemed to provide that without doing a custom build from scratch, it still seems odd that the performance monitoring tools do not show the CPUs as pegged; I guess the tools I am using aren't showing the actual latency within the clock cycles.

I wonder why Premier does not offload that processing power to the GPU, I would think if it did that then the CPU would not be as important. I know Davinci Resolve uses the GPU a lot more than the CPU, it would be great if Premier did more with the GPU.

Bill, I'm confused by your reply, I'm not sure if you are saying the test results are good or bad.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guru ,
May 09, 2018 May 09, 2018

Copy link to clipboard

Copied

Neither Adobe nor Davinci offload decoding/encoding to the GPU's. Both will offload scaling and FX or other frame attributes to the GPU. But neither offload the decoding and frame data to the GPU's and that is where your CPU's fall short for H264 4K. The CPU load will not correlate to CPU processing latency when essentially the frame data is often dependent on other frame data to complete processing. In other words H264 4K is requiring other processing constantly to generate frame data to process subsequent frames which basically means sending many strings of data through the cpu pipeline in specific sequences which is why the CPU clock speed is so important to this process to handle the data in time for realtime playback. All of this process is part of the decoding.

Eric

ADK

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
May 09, 2018 May 09, 2018

Copy link to clipboard

Copied

diesel5598  wrote

Thanks, definitely not what I wanted to hear though. I wanted a 2U rackmount workstation and the Dell was the only one that seemed to provide that without doing a custom build from scratch, it still seems odd that the performance monitoring tools do not show the CPUs as pegged; I guess the tools I am using aren't showing the actual latency within the clock cycles.

I wonder why Premier does not offload that processing power to the GPU, I would think if it did that then the CPU would not be as important. I know Davinci Resolve uses the GPU a lot more than the CPU, it would be great if Premier did more with the GPU.

Bill, I'm confused by your reply, I'm not sure if you are saying the test results are good or bad.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
May 09, 2018 May 09, 2018

Copy link to clipboard

Copied

Your score on the Disk I/O test is the performance you would get from a Hard disk drive.  Why are you exporting the a clunky old hard disk drive when you have in your words a NVMe storage spaces direct RAID system.  What of your storage devices are in this system.

I sure cannot recommend this configuration based on your PPBM test results.  This is a software item which is probably the reason you CPU intensive score is higher than another test of of this CPU.

If you can set up a reasonable sized demo project to download I wil run the project on my i7-5960X

Do you know anything about this Understanding the cache in Storage Spaces Direct | Microsoft Docs

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
May 10, 2018 May 10, 2018

Copy link to clipboard

Copied

Bill, I'm not sure your test is accurate. The test was not run from the Storage Spaces array, it was ran from the C:\ drive which is a single dedicated NVME drive. According to PassMark my NVME drive is performing within it's expected range, so I'm not sure if your test scripts are accurately measuring the disk performance as the following screen shot shows. I am running Premier Pro CC 2018, not sure if that makes a difference.

I think I am just stuck with CPUs that cannot keep up like ECBowen said. I probably should have done a custom build with an Intel i9 using a custom rackmount chassis.  My only hope is that the inbound RAM will make a difference.

PiterSpb, I cannot use QSV because my CPUs do not support it. I did try both enabling and disabling H.264 acceleration and it did not seem to make much difference.

Passmark-Results.JPG

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
May 10, 2018 May 10, 2018

Copy link to clipboard

Copied

diesel5598  wrote

Bill, I'm not sure your test is accurate.

Passmark is not Premiere.  PPBM has been run 1000's of time and helped hundred's of people make Premiere run properly

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
May 11, 2018 May 11, 2018

Copy link to clipboard

Copied

Bill, then I'm not sure there is much I can do about that score. PassMark says my disk is running as it should, if the PPBM score is accurate then for some reason Premier is not using the full potential of the drive so possibly there's a bug in Premier or a driver that is not working properly between Premier and the disk. Either way, I never see any of the drives utilization go over 10-15% so I don't think that's the bottleneck.

My 128GB RAM upgrade arrived today, if that does not fix it then I'll just have to use proxies at times.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
May 11, 2018 May 11, 2018

Copy link to clipboard

Copied

Well I tried my best.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
May 22, 2018 May 22, 2018

Copy link to clipboard

Copied

Bill, All,

  Thank you for your attempts to help. I have a somewhat usable configuration that lets me playback 4K 60FPS Long GOP media from the DJI P4P and GH5 without proxies. Each change below brought slight improvements, I think I'm now at the best it will get.

1) I upgraded to the April 2018 update for Windows 10

2) I do not add my color adjustment layer until all other editing is done

3) I do not color grade any individual clips until all other editing is done

4) (This gave me the biggest improvement) when creating the project I right click a clip that is 4K at 60FPS and tell it to create a new sequence using that clip's settings. Even if most of my media is 30FPS, I use the 60FPS clip for the sequence settings.

5) I upgraded my memory to 144GB of RAM

With all of those changes in place and playback resolution set at 1/4 or 1/8 I can edit the media without having to create proxy files. It still drops frames, and sometimes I have to move the playhead back a few times until it stops stuttering, but it is marginally tolerable. The painful part is when a customer wants changes after the color grades are in place.

If I had to do this over I would have gone with a custom build, 4U rackmount chassis and i9 CPU; but I love the stability of my Xeons. My previous editing workstation was an HP Z800 and in 5 years of use Premier Pro, Photoshop, Lightroom, and After Effects maybe crashed 3 times between them. Most people do not realize a lot of crashes they get is from unstable HW or loading up 3rd party codecs.

I hope that the industry releases some kind of dedicated H.264/265 rendering add in card sometime soon. The fact that these cameras can record and playback this media so flawlessly tells me that a highly customized dedicated decoder board would make a world of difference.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
May 22, 2018 May 22, 2018

Copy link to clipboard

Copied

I was asking both in the camera maker's booths (Samsung, Panasonic, and DJI) and the software for Avid, PrPro, and Resolve at NAB about the long-GOP media. The camera spokes-persons all say they have a custom setup chip on or with some customized firmware setups for their specific needs. Meaning even between cameras within one brand, their media can be encoded differently.

So any hardware-based solution needs to be able to work with rather a wide range of encoding possibilities of the various custom made chips along with customized firmware and processor options in the various cameras.

The software folks all said long-GOP is the grungiest ... crud ... they get stuck trying to sort out. Avid guy wasn't particularly interested in even spending much time on it. "Transcode, my friend ... in whatever app might make something actually editable out of that ... (long pause) ... media." I got the distinct impression he really didn't want to have his internal thoughts quoted.

Resolve people were more interested in talking, but still ... noted that Resolve on most computers will need to be set to use Optimized Media (Resolve's t-code process) for any decent playback for editing, color, or sound.

Adobe engineers were actually more willing to spend time talking about it. But clearly, my impression is that they feel like they're chasing a phantom target. There's something like over a hundred different cameras they are trying to accommodate at any one time. And each seems to come out with new firmware or models every few months that change the encoding settings used. In ever more creative ways. So it's in reality maybe a couple hundred or more encoding settings they're needing to match up at any one moment.

Which ... again ... is why in the aisle-talk, several conversations were about suggesting to Adobe's video managers that MediaEncoder be fitted to 'read' as many camera/device media forms as possible to make t-codes from many of them, and PrPro perhaps be built around using a few major editing codecs to get both better playback and stability while editing. Which might make more sense in general. And just give up on trying to keep up with the amazingly creative folks storing camera media on cards.

Intel's QuickSync on some of their CPU's is supposed to be faster at handling H.264 than the standard circuits of the CPU, but I don't have any information on how well that works. So there is a start at least on the hardware side. Would sure be nice if there was a lot more capability in that hardware though.

Neil

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Jun 01, 2018 Jun 01, 2018

Copy link to clipboard

Copied

Neil,  I would be curious to see if Final Cut Pro found a way to improve their handling. I've seen reports that Long GOP media works better on their platform. Of course Apple makes all of their hardware so they can highly optimize their software for their hardware.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines