premiere not using resources?

Premiere is constantly bogging down on me, and does not use ANY available resources.
I'm working in a 1080 timeline, with 4k footage.  My specs are:

2x Xeon 2670's (16 cores)

40GB ECC ram

PCIE scratch disk (2000mbps)

Separate System drive raid 0(700mbps)

External Raid 0 HDD for files (300mbps)

GTX 1080 GPU

Yet, playback when using Neat video is 1-2fps AT BEST, usually it will just freeze.  If I use something like "Cosmo" from magic bullet, I'll get 10fps.  I have never seen my CPU used at more than 20% during playback or export.  Hard drives are never more than 5%, and GPU is at 20% and may fluctuate up to 33% for a brief second.  My RAM usage sits at 50% most of the time.  I have zero idea where I'm bottlenecking.  I honestly had the same performance with my Macbook pro.  I'm aware that neat video is a beast, but I see no benefit in the better specs of the Xeon machine w/ a 1080, vs a mobile i7 with a mobile gpu.

Would any upgrade help me?  I'm totally willing to throw money at this issue.