PP 2015/17 Render / Export Low GPU and CPU usage - (Pictures / Proof)

Report · Nov 09, 2016

Hi Everyone

So I have been having issues with Premier Pro, After Effects and Encoder on my new PC . This seems to be a big issues as I see this problem on forums and discussion boards across the internet and while some have fixed it or made it better there is no definitive global fix. It seems to get tossed up to hard drive performance which I am stating is false. Unless I am proven wrong - I hope my findings will help further push for a solution

This happens equally with 2015.3 and 2017. I do not use 2015.5 (too many bugs but same performance issue)

-- Computer Specifications ---

HP Pheonix Envy 860
Fresh CLEAN Windows 7 Professional 64x Install 10/22/16 - Fully Operational 10/29/16 (Did not use HPs Recovery Disc set)
i7 5820K 6 Core 12 Threads (O.C. 3.6Ghz) - Liquid Cooled
24GB Ram 2133 Mhz DDR4
NVIDIA GTX 960
2x 2TB Seagate 7200 RPM 3.5" HDD SATA 3
- - C: 1x OS / Programs
- - E: 1x Media Storage
1x 750 GB Toshiba 5400 RPM 2.5" HDD SATA 2
- - G: Export HDD
INSTALLED ADOBE VIDEO SOFTWARE
- Premier Pro 2017
- Premier Pro 2015.3
- Encore CS6 (Uninstalled Premier Pro CS6 - Only needed Encore)
- After Effects 2017
- After Effects 2015.3
- Encoder 2017
- Encoder 2015.3

I am making a case as people always say that its the amount of hard drives in the computer or the fact they are 5400rpm vs 7200rpm or your not using a SSD for your OS, or your Adobe Cache is not on the correct drive and adobe heavily utilizes this cache during exports (which I found to be false on my PC)

So I ran a battery of comparison test and more to come... I compiled all the tests down to 4 below.

-- 1ST SERIES OF TESTS --

Specs during test
2x 2TB 7200 RPM HDD
- C: OS / PROGRAMS / CACHE

- E: MEDIA STORAGE / EXPORT
NO 750GB HDD

Original Video File: 6.5minute 1080p 50Mb/s bitrate. MP4 XAVC-S

Exported Video File: 6.5 Minute 1080p 40Mb/s bitrate H.264 - No effects
CUDA HW Accel PROCESSING ACTIVE

Exporting video file from Premier Pro / Encoder

According to Windows Resource Monitor

Adobe READS from E: MEDIA STORAGE @ 4.5MB/s
Adobe WRITES to E: MEDIA STORAGE at 2.5 - 3.5 MB/s.
CPU 45%-50% Spikes of 60%-70%
GPU 1%, 5%, 25%, 50%. Spikes of 100% every few seconds for a split second (Varies to much) Average I would say is 20%

Changes and Adjustments

Moving Cache from one drive to another made ZERO difference.

Exporting to one drive or another made ZERO difference

-- 2ND SERIES OF TESTS --

Specs during test

2x 2TB 7200 RPM HDD
- C: OS / PROGRAMS

- E: MEDIA STORAGE
- G: EXPORT HDD (Added to test if an additional drive would help) Specs for drives at TOP of post

- F: 256GB USB THUMB DRIVE (USB 3.0)

Original Video File: 6.5minute 1080p 50Mb/s MP4 XAVC-S
Exported Video File: 6.5 Minute 1080p 40Mb/s H.264 - No effects
CUDA HW Accel PROCESSING ACTIVE

CHANGES & ADJUSTMENTS

Moving CACHE between 3 drives had ZERO effect. In fact, No cache activity was reported during exports when one drive was strictly used for cache only

Image Below shows export writes at 3.3MB/s to MEDIA STORAGE E: (Media reading from E: as well) while CACHE was moved to G: - Notice....G: has ZERO activity during exports in this test

Test 1 - Export from Premier Pro to G:

Exporting video file from Premier Pro / Encoder

According to Windows Resource Monitor

Adobe READS from E: MEDIA STORAGE @ 4.5MB/s
Adobe WRITES to G: MEDIA STORAGE @ 3.5 MB/s.
CPU 45%-50% Spikes of 60%-70%
GPU 1%, 5%, 25%, 50%. Spikes of 100% every few seconds for a split second (Varies to much) Average I would say is 20%

Test 2 - Export from Premier Pro to G: while Copying a 34.8GB video file FROM E: MEDIA STORAGE to USB THUMB DRIVE (USB 3.0)

Exporting video file from Premier Pro / Encoder

According to Windows Resource Monitor

Adobe READS from E: MEDIA STORAGE @ 4.5MB/s
Adobe WRITES to G: MEDIA STORAGE @ 3.5 MB/s.
FILE COPY WRITES to F: @ 80+ MB/s
FILE COPY READS from E: @ 80+ MB/s
CPU 45%-50% Spikes of 60%-70%
GPU 1%, 5%, 25%, 50%. Spikes of 100% every few seconds for a split second (Varies to much) Average I would say is 20%

Test 3 - Export from Premier Pro to G: while Copying a 34.8GB video file FROM USB DRIVE (USB 3.0) TO G: at same time

Exporting video file from Premier Pro / Encoder

According to Windows Resource Monitor

Adobe READS from E: MEDIA STORAGE @ 4.5MB/s
Adobe WRITES to G: MEDIA STORAGE @ 3.5 MB/s.
FILE COPY WRITES to G: @ 80+ MB/s (Windows Copy Window says 200+MB/s - confusing)
CPU 45%-50% Spikes of 60%-70%
GPU 1%, 5%, 25%, 50%. Spikes of 100% every few seconds for a split second (Varies to much) Average I would say is 20%

From these test, I find that the hard drives are not at fault and 3 drives make little difference than 2. If adobes encoder worked the way it should, more drives might make a difference, but in this situation, adding another 3rd drive did not make any major difference...only about 1MB/s write/read. I was able to copy and export to the same drive which made no difference in encoding time or encoding read/write speeds. Which tells me the HDD and SATA system is not choking.

Moving CACHE from one drive to another made ZERO difference. Adobe Encoder seems to not use CACHE during exports. Probably only for Renders.

Can anyone tell me what the heck is going on? I bought this PC thinking I would get insainly fast exports as I have very important TV Show work to do and I can barely preview edited work with minor effects in real time or export in any reasonably timely manor. I also wanted to export in 4K.....it takes unbelievable amount of hours to export in 4K which it shouldn't.

I am going to compare my results to more test I will perform on my i5 Laptop I used for encoding before.

Report · Nov 09, 2016

GPU usage most of the time is between 1%-7% at any point in time. Sometimes it does get to 25% or even 50% for a split second but doesn't last. Once in awhile it my peak 100% for a split second. Most of the time the GPU is idle and the CPU is under 50%

Report · Nov 09, 2016

PrPro doesn't use a GPU much during export/rendering except for things on the "GPU accelerated" list ... the Lumetri panel corrections I think are on that, I know say re-sizing is ... some other things. If you're not running those things, the GPU isn't involved. The rest of encoding is a CPU process.

As to the rest of your questions, I think RoninEdits or Bill Gehrke would have better thoughts ...

Neil

Report · Nov 10, 2016

Hi there thanks for the reply

I do understand that not everything can run thru the GPU and its mostly thru the CPU. Though if the GPU sits idle, I would expect my CPU usage to increase drastically at least to 90% if not 99% during exports / render. Only time I would want to see my CPU lower is if the load is handed off to the GPU.

Export Times - No Effects or Editing

6.5 Min 1080p 50Mb/s Clip to 6.5 Min 1080p 40Mb/s = 10 Minutes
1 Minute of the same 6.5 min clip to 1080 40Mb/s = 1 minute

My main concern is the very slow read and write speads. If I were to export a 1 hour 1080p clip at 40mb/s it would take 1-2 hours....thats not right.

Adobe READS from E: MEDIA STORAGE @ 4.5MB/s
Adobe WRITES to G: MEDIA STORAGE @ 3.5 MB/s.

Only way I can get faster writes is Overclocking the CPU even more. Pushing it to 4.6Ghz on a HP China branded Motherboard - doesn't make for stable speeds but was able to achieve 5MB/s write speads which eventually slowed down to 4.5 MB/s write speeds. That is with adobe using 50-60% of CPU

I see videos online exporting MUCH faster...1 minute video exporting in 8 seconds or so..not sure there exporting settings....

Report · Nov 10, 2016

Bill Gehrke (over at the Tweaker's page) has a lot of info, most still quite useful, on building/setting up a "balanced" system. This is where much of the typical multi-disc information comes from. There are many factors involved, including what was once called "bus speed" of the motherboard (I've been told that's old terminology but I don't know the new one), the relative cycle-frequency of the CPU & RAM, the in/out capability of the mobo and particular drives involved, frequency rate of the CPU, probably twenty other things.

And it's a shifting target. Bill has found that the new Samsung T3 SSD external drives can sustain read/write speeds at or above what many to most internal SATA III drives can ... external SATA (eSATA) and Thunderbolt had been considered the only external drive connections really usable for speed gains within the Adobe digital video/audio programs (DVA's) but ... that's not necessarily true anymore.

There's a massive amount of data involved in the encoding process ... and figuring out how to setup a machine so that the data flies at best possible speed is not always an obvious task.

Realistically, the best place for this discussion is the

https://forums.adobe.com/community/premiere/hardware_forum

which is where those with the best knowledge of these matters "hang out". You'll get more responses there, and from those with better knowledge.

Neil

Report · Nov 10, 2016

Its just the fact that my HDD and Sata interface is not choking since I can copy one large file to the same drive im exporting to at the same time and the copy reaches 80+MB/s while adobe is sitting at 3.5MB/s I don't get it.

I've looked on the tweakers page but I think there is more to it. Since adding another hard drive did pretty much zero for increase in speed.

I have $500 in parts in a shopping cart online ready to order...but I need to know if its well spent 500 for HDD's or look at different parts or software / settings.

Report · Nov 10, 2016

the hdd speeds while exporting from premiere isn't a sign of the hdd's limiting premiere, but a sign of how little premiere is using the hdd's. you point this out very well by being able to copy files in the background with little to no impact on premiere's performance. overclocking achieved higher disk speeds as a byproduct of speeding up premiere via the cpu, allowing premiere to work faster.

there are two main reasons why premiere might be using less than 90% cpu usage. either a hardware bottleneck or software bottleneck. for hardware you have already ruled out the hdd's and gpu, so you could check the cpu with cpu benchmarks or stability software and make sure the cpu is at 99-100% and cpu temps are ok. prime95, cinebench, real temp and core temp are some free programs to check that out. the cinebench r15 score for your cpu should be around 1200...

if the hardware is ok, then its something with the software. premiere has problems when it comes to using lots of cpu cores, and its possible premiere just isn't using all 6 cores in your cpu to max out cpu usage. this is often caused by the media codec or by certain effects on the clips or timeline. it looks like in some of your tests you aren't using any effects, so that would leave the media codec. you can try transcoding to cineform or dnxhd/r or other codecs and see if any improve performance and cpu usage. the h264 codec inside the xavc-s isn't going to be an edit friendly codec, cpu's are poor h264 decoders.

Report · Nov 10, 2016

I forgot to mention that premier pro is using all 6 cores and all 12 threads. i havent tested anything yet but ill try to get to it tmro afternoon

Report · Nov 11, 2016

That is what my task manger shows you above

Report · Nov 11, 2016

I meant on my PC haha during this issue. I am going through some of the testing you suggested right now.

Report · Nov 11, 2016

TEST FOOTAGE From Sony Action Cam - Footage I used for testing above ^
General

Format : XAVC

Codec ID : XAVC (XAVC/mp42/iso2)

File size : 2.45 GiB

Duration : 6 min 45 s

Overall bit rate mode : Variable

Overall bit rate : 52.0 Mb/s

Encoded date : UTC 2016-11-02 10:25:17

Tagged date : UTC 2016-11-02 10:25:17

Video

ID : 1

Format : AVC

Format/Info : Advanced Video Codec

Format profile : High@L4.2

Format settings, CABAC : Yes

Format settings, ReFrames : 2 frames

Format settings, GOP : M=1, N=12

Codec ID : avc1

Codec ID/Info : Advanced Video Coding

Duration : 6 min 45 s

Bit rate mode : Variable

Bit rate : 50.0 Mb/s

Maximum bit rate : 60.0 Mb/s

Width : 1 920 pixels

Height : 1 080 pixels

Display aspect ratio : 16:9

Frame rate mode : Constant

Frame rate : 50.000 FPS

Standard : PAL

Color space : YUV

Chroma subsampling : 4:2:0

Bit depth : 8 bits

Scan type : Progressive

Bits/(Pixel*Frame) : 0.482

Stream size : 2.36 GiB (96%)

Encoded date : UTC 2016-11-02 10:25:17

Tagged date : UTC 2016-11-02 10:25:17

Color range : Limited

Color primaries : BT.709

Transfer characteristics : IEC 61966-2-4

Matrix coefficients : BT.709

***Footage from Another Sony Cam - FDR-AX100***

General

Format : XAVC

Codec ID : XAVC (XAVC/mp42/iso2)

File size : 204 MiB

Duration : 17 s 17 ms

Overall bit rate mode : Variable

Overall bit rate : 101 Mb/s

Encoded date : UTC 2016-11-02 13:04:27

Tagged date : UTC 2016-11-02 13:04:27

Video

ID : 1

Format : AVC

Format/Info : Advanced Video Codec

Format profile : High@L5.1

Format settings, CABAC : Yes

Format settings, ReFrames : 2 frames

Codec ID : avc1

Codec ID/Info : Advanced Video Coding

Duration : 17 s 17 ms

Bit rate mode : Variable

Bit rate : 99.0 Mb/s

Maximum bit rate : 100.0 Mb/s

Width : 3 840 pixels

Height : 2 160 pixels

Display aspect ratio : 16:9

Frame rate mode : Constant

Frame rate : 23.976 (24000/1001) FPS

Color space : YUV

Chroma subsampling : 4:2:0

Bit depth : 8 bits

Scan type : Progressive

Bits/(Pixel*Frame) : 0.498

Stream size : 201 MiB (98%)

Encoded date : UTC 2016-11-02 13:04:27

Tagged date : UTC 2016-11-02 13:04:27

Color range : Limited

Color primaries : BT.709

Transfer characteristics : IEC 61966-2-4

Matrix coefficients : BT.709

Report · Nov 10, 2016

MODERATORS: Could you move this to the HARDWARE section of Premier Pro please?

Report · Nov 10, 2016

Well if you really want to see your CPU and GPU usage measured you will have to go back to Premiere Pro BenchMark (PPBM) and run it. Just for you I "crippled" my i7-5960X to look like your i7-5820K. So running with 6-cores at 3.6 GHz here is what happens when the CPU intensive MPEG2-DVD timeline is exported without GPU assistance:

.

Notice 6-cores at 3.57 GHz and 100% utilization

Next, when you enable the GPU acceleration with my GTX 1060 SC at stock speed, here is 99% utilization

See the GPU Load above.

Now for the scores.

"24","81","23","414", Premiere Version:, 11.0.0.154, simulated i7-5820K at 3.6 GHz with the GTX1060 SC

TBA more comparison scores and explainations

Sorry have to run will finish this later this evening--I have to run right now.

Report · Nov 10, 2016

You guys are awesome. This is the type of "other possibilities" I'm looking for. I"ll check all this out and get back to you on my findings.

RoninEdits

Thanks to both of you!

If anyone else has other information please post it.

Report · Nov 10, 2016

Continued from above!

Now for the scores. All Win 10, Exports to an M.2 950 Pro, C:\ = SATA III 840 Pro, GTX 1060 @ 2400 MHz Memory Clock, 64 GB RAM at 2133 MHz

"24","81","23","414", Premiere Version, 11.0.0.154, simulated 6-core i7-5820K at 3.6 GHz,

"24","55","19","250", Premiere Version, 11.0.0.154, 8-core i7-5960X 4.5 GHz,

The major CPU intensive score is the fourth number It would be interesting if you run the test to see how your numbers with a real i7-5820K.
As you can see in the second and third scores are GPU accelerated scores but as you can see it is also somewhat CPU dependent. The third number is the same timeline that was used to measure the CPU-only fourth number but this time GPU accelerationT is turned on. The second number is a seven-layer timeline exported to h.264 with a wide variety of media and is slightly more CPU dependent.
The first number is the time it takes to export a very disk intensive timeline since I am exporting all test to my M.2 Samsung 950 PCIe x4 and does not have any significance here.
Remember these timelines were designed to demonstrate hardware efficiencies and do not represent what you might expect in real life. Also all scores are +/- 1 or 2 seconds because the clock in Windows is 1 second granularity

Report · Nov 11, 2016

Before this test I reset my BOIS to factory spec - all this from here on out will be at stock CPU clock speeds 3.3ghz

Test was on Premier Pro 9.2

Premier Pro Bench Mark Project crashed Premier Pro's export ("Error compiling movie, Unknown error") on the H.264 timeline. I did some troubleshooting and found the .R3D file called "L004_C171_0923A2_001.R3D" does not work so well. I don't know if its corrupt...or if im missing a codec or for another reason, but it will not show that video file at all in the preview, just a black screen. If I remove the first insert of that file from the time line, PP will export till it hits the next same file.

EDIT: I noticed v11 doesn't have that issue. Im going to try and run PPBM10.3-DE on v11 even tho it has to "convert it".

What do you suggest? I will re run the tests again and get the actual numbers after we figure this out

FYI - For now here is what I got
Disk Test was maxing out at 91.5MB/s write speed to my EXPORT Drive (CPU & GPU mostly idle)

MPEG2-DVD 1st Test - GPU 100% - Write 1.5MB/s

MPEG2-DVD 2nd Test - GPU Idle - CPU 100% - Write 0.21MB/s

Report · Nov 11, 2016

Something is wrong in your computer, I have never seen that error before with several thousand runs. Do you have any non-Adobe codecs installed?

Report · Nov 11, 2016

So seems there is another issue... PP ver 9.2 and 10.3 both will not read R3D files. PP ver 11 will play it just fine though

Even after reinstalling PP 9.2 it still doesn't play R3D files.

Report · Nov 11, 2016

"Error compiling movie, Unknown error" seems to be one of the more common problems with premiere lately. there are several threads with different causes and possible fixes. there was a nvidia driver problem with r3d footage a while ago, you could try installing the latest nvidia driver.

Adobe Premiere Pro Forums - Search: Error compiling movie

this picture shows workload on all cores/threads, but premiere only has around 3-4 cores worth of workload and the workload is being spread to all cores/threads.

this picture shows premiere having 6 or more cores worth of workload and all available cores/threads are constantly maxed.

Report · Nov 11, 2016

I did have ffdshow installed as I was trying something out but I removed it....

I just reinstalled PP 9.2 and it still doesn't read the R3D footage

Report · Nov 12, 2016

I ran the test on my laptop just to see what the numbers would be...but I keep getting this error when running the script. Is there a certain location I need to put the script?

Report · Nov 12, 2016

The script is looking for your exports, there is no way we could know where they are located so it just looked in the project file. If you want to move the exported files you must CUT and paste. If you copy and paste the time stamps get changed

Report · Nov 29, 2016

Hi Guys,

Sorry I haven't updated this in awhile. Well the test projects seem to work fine in PP 2017 without the problem with the red video file. I just haven't run the scripts and been busy.

Lately I have been doing a lot of work in Adobe After Effects and that too, knowingly it has a lot of work to do, uses 10-30% CPU when encoding directly to a MP4 using H.264. So I decided to open up multiple instances of AE using the " -m " attribute in the shortcut path. After opening 2 up and rendering with multi-machine settings (1 pc) and rending to JPEG sequence I was able to cut the time down to just over half and CPU usage was around 70-90%. I tried again with 3 up and now the encoding time went down to under half (23min for 1:05 (min/sec)) project with lots of effects from 1 hour and CPU usage is 97-100%. Memory seems to be less utilized too. 60% vs 80%.

If there was a way for Adobe to give the option of encoding to JPEG sequence using multiple instances of media encoder and then encoding to the proper video format automatically.....that would drastically improve encoding times. Seems like a work around but it could work.

Thoughts?

Report · Nov 29, 2016

Not sure if I can open multiple instances of PP and have it work the same so I'll probably import my PP projects into AE and render them to JPEG sequences then to whatever video format I need.

Maybe there is an external script that can help with this....multiple simultaneous renderings/encodings

Report · Nov 29, 2016

I read about how adobe does it. it looks at a watch folder to see what frames haven't been processed yet. It only works on image sequences because there's no way to know when a video frame is done being processed as some are not I frames. either you would need a really sophisticated algorithm or a way to have media encoder see the same folder across several virtual machines. theoretically should work with premiere by chopping up the timeline into equal in/out points as image sequence.

-----------

also,

I read a nice review that said the fastest exports for

AE was 6GB RAM per core.

" if you are exporting to H.264 1080p, you would ideally want a CPU with about four or six CPU cores!."

https://www.pugetsystems.com/labs/articles/Adobe-Premiere-Pro-CC-2015-Multi-Core-Performance-Update1...

Try starving premiere of excess cores/threading if you don't have RAM=Cores*6

anothe article said that premiere can't see past 24 cores anyway.

and some effects are single threaded(len distortion), so add them later after final export.

and finally,

----------

- The Super Fast Premiere Export Guide -

1. run creative cloud software update

2.launch media encoder before opening premiere(there's a single core bug if its opened after)

3. cleane the cache setting, delete temp files

4.update neatvideo and run its optimize tool.

5. check if cpu's are parked. The utility is called ParkControl

or select High Performance profile in Control Panel-Power management

6. change available memory by changing the rendering optimization preference to Performance from Memory or visa versa if you have large image formats.

7. shrink large images to sequence size before importing into premiere, stay away from PNG, they are not thread safe!

8. don't use lumetri

9. enable smart rendering 4x-12x faster render

http://blogs.adobe.com/kevinmonahan/2012/10/11/smart-rendering-in-premiere-pro-cs6-6-0-1-and-later/

10. don't use adjustment layers

11. don't use 64 cube luts

12. don't use dynamic link with AE projects. render them out first, expecially warp stabilized shots