Extremely slow rendering in timelines and exporting

Hi,

(I'm relatively new to Premiere Pro so please excuse any potential misuse of words, etc)
So the problem I am sadly facing is following: When trying to preview any sequences (or timelines for that matter) that had chances made to it (pretty simple things) it becomes impossible to view the timeline let alone to export. When trying to "Render In to Out" or "Render Selection" it takes an absurd amount of time. Roughly 20 seconds of MP4 4k footage @ 25fps take 30-60 minutes (or sometimes even longer, yes, even after waiting for the Render to progress a bit).

Roughly the same time is displayed when trying to export using Premiere Pro, yet Adobe Media Encoder gets the job done in mere seconds. 

I have Hardware that should certainly be able to handle massive workloads which makes this even more confusing.

Hardware is:

• CPU: R7 5800X @ 4.2Ghz Base / 4.8Ghz Boost
• RAM: 64Gb (54Gb allocated to Adobe) Corsair Vengeance @ 3666Mhz
• GPU: NVidia Aorus 3080ti

In addition to that I have multiple NVMe SSD's installed. I should mention that all of the above mentioned hardware is custom overclocked, but thoroughly stress-tested so that shouldnt be a problem.
When trying to Render something, my CPU/GPU usage barely goes up at all.  

Note the Highlighted areas

Meanwhile Rendering with the Media Encoder it looks more like this:

Note that now not only are the CPU and GPU usages much higher but also the the clock-speeds of the GPU is way higher than before. While the clock speed of the CPU stays roughly the same, this time all the cores are used equally.

If it were only for the exporting, Id be totally fine using the Media Encoder but being unable to navigate or preview the timeline really makes Premiere Pro unuable for me. I have tried different things already in hopes to be able to fix it but nothing has worked yet.

Thanks for any help already!