Premiere Pro CC (2018) bottleneck

[xperiense]

Hello guys,

So i've got 3 questions:

1. Is everyone experiencing CPU bottleneck with Premiere?

We mainly edit 4k h264 footage from DJI X5 RAW camera @ 60Mbps

I know premiere is more CPU intensive in every way but i hate it so i have to make the rig work better.

With the GTX 1080 i 've found that encoding times and render previews worked faster for a 1080p timeline and export.
With the Radeo Pro Duo i 've experienced kind of slower exports (without resetting profiles) and render previews to about 20 seconds difference.

Though i think that i get slightly better timeline editing performance with the Radeon.

I really don't care about exporting time (reasonable times of course), my main concern is the real time timeline editing performance which i believe that's what all benchmarks should also include but mostly they don't. Also not including AMD professional cards is also very suspicious.

What i 've seen within premiere is that the CPU is hitting 100% load quite often especially on rendering and exporting while the GPUs just occasionally being utilized for a fraction of a second at about 5-10%.

2 .Is there a way to eliminate the bottleneck on the current configuration?

After effects can't fully support the Radeon Pro Duo though i get really good realtime performance with particles and stuff. The radeon also performs far better than the GTX 1080 in Photoshop 3D mode while one of them GPU 0 being utilized at 45% max and the cpu stays at 20% max.

My next question is:

3. Will an AMD Threadripper 1950x solve my problems with Premiere and help me edit without any glitches or slow framerates in timeline based on my current config or does the problem rely on the DJI footage no matter what cpu i add?

EDIT: I know that using proxies would be a way to go but i'm trying to understand if i can get it to work without them.

My instinct says it will but i need some help to be sure.

My configuration:
X99-Ultra Gaming
Core i7 6850k @ 4.2Ghz

RAM Corsair Vengeance 32GB 3200Mhz @2933Mhz
Samsung 850 Evo Sata SSD (Boot)
Samsung 960 Evo Nvme (Scratch & Projects)
Samsung 950 Pro Nvme (Footage)
GTX 1080 Gigabyte G1 Gaming 8GB (previous)
AMD Radeon Pro Duo (Polaris) 32GB (current)

[Bill Gehrke]

1. No
2. Yes
3. Possibly

There is another way. Transcode to a media form that is easier on your CPU.  Your media is a long GOP media which requires an awful lot of CPU to generate each and every frame.  Cineform media which is much easier to edit is the way to go.  Try the GoPro free version.    It is a very high quality industry standard format, the only real disadvantage is that because you have each and every frame is there, your media files will be much larger.  I do not have a link but just search these forums for help

[RjL190365]

Unfortunately, Bill, the Cineform encoder is no longer available for download directly from GoPro, as the program that included it (GoPro Studio) has been EOL'd three months ago. And the last version of GoPro's Quik desktop app that included GoPro Studio was version 2.3, which is also no longer available for download. Beginning with version 2.4, GoPro Studio is no longer included with Quik. And although GoPro will continue to support and update the Cineform encoder, no current or future GoPro software products will include it.

By the way, Premiere Pro CC 2015 and later (including CC 2018) includes a Cineform encoder, and that encoder should have been installed as part of the installation. Plus, since at least CC 2017, Premiere Pro should have given the OP the option to use proxies or transcode upon import (ingest). Cineform MOV presets are included with this feature.

[Bill Gehrke]

Thank you Randall for bringing me up to date.

Also I guess that I should recommend that  xperiense  run my Premiere Pro BenchMark to see if it is tuned properly

[xperiense]

When i try to run the vbs it says file not found.

I opened the project from both nvme drives and it showed something really interesting though weird.

The 960 Evo 256GB is plugged directly on the motherboard slot.

The 950 Pro 512GB is plugged in a Pci-x controller.

When i run samsung magician benchmark i get

960 Evo

Sequential (MB/s)

Read 3,271

Write 1,600

Random (IOPS)

Read 398,591

Write 280,984

950 Pro

Sequential (MB/s)

Read 2,591

Write 1,536

Random (IOPS)

Read 325,098

Write 104,785

Disk Test

1. When i opened the project from the 960 Evo

Export to 950 Pro took 25 seconds (Speed steady at 1.5GB/s)

Export to 960 Evo took 78 Seconds (Speed starts @350MB/s and goes down to 235Mb/s)

2. When i opened the project from the 950 Pro

Export to 950 Pro took 25 seconds (Speed steady at 1.5GB/s)

Export to 960 Evo took 78 Seconds (Speed starts @350MB/s and goes down to 235Mb/s)

I tried swapping the drives but the controller can't recognize the 960 Evo

Anyway now after all the plugging i tried exporting to 960 evo and it slowed down even more at 85 seconds

MPEG2-DVD Opened from 960 Evo (Warning replaced Fonts)

Export to 960 Evo - 32 Seconds (CPU @ 28%)

Epxort to 950 Pro - 31 Seconds (CPU @ 28%)

H.264 Test Opened from 960 Evo (Warning replaced Fonts)

Export to 960 Evo - 57 Seconds (CPU @ 65% - 100% mostly at 100% - GPU Max 5% on both GPUs)

Epxort to 950 Pro - 59 Seconds (CPU @ 100% - GPU Max 5% on both GPUs)

MPEG2-DVD Opened from 950 Pro (Warning replaced Fonts)

Export to 960 Evo - 32 Seconds (CPU @ 27%)

Epxort to 950 Pro - 31 Seconds (CPU @ 27%)

H.264 Test Opened from 950 Pro (Warning replaced Fonts)

Export to 960 Evo - 59 Seconds (CPU @ 45% to 100% mostly at 95% - GPU Max 5% on both GPUs)

Epxort to 950 Pro - 58 Seconds (CPU @ 45% to 100% - GPU Max 5% on both GPUs)

[Bill Gehrke]

"The 950 Pro 512GB is plugged in a Pci-x controller.".  I assume you really meant PCIe bus with an adapter card,  What one?  What PCIe slot?

"When i try to run the vbs it says file not found." This happens when you run the script from other than the location of the export files..

i can see no logical explanation for those weird results.  The only unusual situation is that I have never seen operating with both a CUDA GPU and an AMD GPU at the same time.  If you could pull out that Radeon for a test to see if that disk exporting weird action goes away (doing that frees up PCIe lanes).

[exile1972]

Your source footage being H264 is the bottleneck and not Premiere. Transcode that footage to something less CPU intensive (like DNxHD) and your editing should be smooth.

[xperiense]

I don't have both cards in there. I wrote about the GTX 1080 because i did use it extensively until now and i know how it performs on this system. Only the Radeon Duo Pro (Polaris) 32 GB is on the PCI-E 1 x16 right now. Then the Adapter for the NVME Drive 950 Pro is on PCI-E 2 X8, PCI-E 3 is Free because it shares bandwidth with PCI-E 1, PCI-E 4 X8 10Gbe Lan.

It seems there is a problem with the write on the 960 Evo. I tried copying the Disk Test.avi file.
960 Evo -> 950 Pro = 1.5GB/s steady throughout the copy
950 Pro -> 960 Evo = Starts at 1.5GB/s and after 10 seconds it drops to 330MB/s and stays there throughout the copy.

I will try moving the 950 Pro on the motherboard slot to confirm that the problem is on the 960 Evo and not the motherboard but i think the latest evo firmware might have done the trick.

[Bill Gehrke]

I think you are thermal throttling your 960 EVO, a condition I have not seen on my M.2 devices as you can see form my storage test results. I do have a small fan blowing air across my 960 PRO 1 TB motherboard mounted M.2 and then the air goes past the adapter board mounted 960 PRO 512 GB M.2.  For the following test I pu

t my 960 EVO on another adapter card and it also has a littlle air flow past it.  As you can see from the chart from my PPBM website plot the export time for those drives I never have seen any throttling on my units.

So I went one step farther and found a way to monitor the temperature of the M.2 drives it is HWiNFO below you will see below that under the Disk I/O export my 960 EVO reach a maximum of 62 degrees C

It took awhile to find these numbers in HWiNFO but if you select Sensors it helps

[xperiense]

The Write Drop starts much earlier than when the temperature starts to rise. But i does get higher and higher while trying to copy. Can you confirm the firmware you are using. I have the 3B7QCXE7

[Bill Gehrke]

Mine is the original (never upgraded) 287QCXE7

We do not know where that temperature sensor is so one of the several chips on that device could be causing the problem but it may have some thermal lag to reach the sensor.

[mikeklar]

Careful, don't upgrade firmware using Samsung's Magician. 

I just did by mistake for both the 950 Pro and one of the 960 Pro's.  As you can see the Read / Write speeds have dropped !!!

Also, Samsung is impossible to get hold of ... 

Bummer

[mikeklar]

Looks like last night's escapade wasn't as bad as shown above

It appears the firmware update required a reboot and due to the late hour I only did so this morning...

Note; when I mentioned the firmware update was done unintentionally, it was due to the "refresh" icon on the top left of Samsung's Magician's window.  It isn't a "refresh" - unless one accepts refresh means updating the firmware

Below are the results as of now...

Also attached is a screen clip showing the temperature of the 960 Pro as it was running the "4K Write" cycle.  The actual temperature reached was 51 degrees Celsius.  It's the line at the very bottom, which is partly cut off...

[RjL190365]

In this instance, the newer firmware appeared to have reduced the sequential read speeds slightly.

[mikeklar]

Yes, that became evident after the original speed tests with the firmware as it came out of the box.

There was also an issue as soon as the firmware was updated the ASUS mother board no longer recognized bot the 950 and 960 as PCIe Gen 3 x 4.  This was eventually resolved with an ASUS Bios upgrade.  However, there was that loss in speed you noted.

Of interest, not upgrading the 950 or 960 bios allowed these devices to function without issues...

Cheers

Edit

It may be the NVMe driver, see the clip below, out of the three Samsung 9xx pros only the one with the higher Read rate is not shown as a NVMe device -

[Bill Gehrke]

I am see you Speccy results above and i believe it is the Speccy tool just has not caught up with the newer technology.

Below that what is the ambient temperature with no load on that 960 EVO and where is is mounted?

Edit

I really think you should send that 960 EVO to Samsung for testing!

[xperiense]

The 960 Evo is mounted onto the motherboard slot. At idle the temperature is at about 42 C.

I have already send Samsung my specs and configuration with some screenshots where a simple file copy drops and i am waiting for their reply. Although the Evo is not my main concern here. Unfortunately based on how i've seen it work so far i have to jump to Threadripper at some point

[Bill Gehrke]

That 960 EVO ambient temperature is about 10° C hotter than my motherboard mounted unit, Of course I said I have a sperate fan at that point.

Well jumping to Threadripper will not solve that specific problem.

I see above that you have run 3 out of the four PPBM tests the last one is the export of the MPEG2-DVD timeline without GPU Acceleration i.e. CPU only.  What is your score on it so I can see how much improvement might be possible?

[xperiense]

It took 76 seconds to export with CPU only

[Bill Gehrke]

Sorry but something must have been set up wrong that number is impossible

An i7-6850K at 4.4 GHz normally scores about 300 seconds on that test

[xperiense]

It must be the Space Shuttle!

I open the PPBM12-DE.prprj from "New Folder" on my 950 Pro
i select to the MPEG2-DVD test timeline.

I go to FIle -> Export Media and i leave everything untouched. I add the export to the queue and select Render "Software Only"
Am i doing something wrong?

Edit: I used the export directly without media encoder but changed the project settings to software only and it took the same amount of time to export.

[Bill Gehrke]

Follow these instructions:

I guess something changed in CC 2018 to see the Step 3 instructions.  I had to advance the CTI in this window, previously you could advance it in the real timeline and it would come up when you did the Ctrl M

[xperiense]

Actually Maximum Render Quaitiy didn't check itself.

Now that i've checked it it takes 313 seconds on CPU only and 32 seconds with GPU Acceleration and the CPU stays at 26-29% load

Edit: I redid the h264 test it got 73 seconds.

Note: i currently have on open Teamviewer session on idle and chrome opened while doing the above tests

[Bill Gehrke]

So this what your Output.csv file would look like:

"25", "73","32","313",

These are all fairly good numbers, both the 73 seconds and the 313 seconds possibly would go down slightly with fewer processes running.  Your GPU assisted MPEG2-DVD s core seem high to me as I tested a Superclocked one and got 15 seconds and would expect about maybe 18-20 seconds   Was it running 99% usage?

See if you can get that idling temperature down.  I have never seen thermal throttling in my system. As I said you may have to install a small fan to cool you M.2 devices

[xperiense]

The GPU assisted MPEG2-DVD takes 32 seconds and the CPU stays at max 29% load.

I will install the gaming drivers for AMD Radeon Pro Duo as well and test.

Edit: Seems like the only card that doesn't have the Gaming feature is mine! Yey!