Adobe media encoder using 100% GPU on 3D while there's nothing thats 3D and barely using CPU

I guess you could need a bit more RAM. Can you open task manager and see how the app behaves when rendering the project?

Thanks for posting that screenshot, it was really helpful. You can see in the bottom of AME that you're using Hardware Encoding and the Nvidia codec. This means that the encoding is happening on the GPU and your Task Manager is showing that your GPU is the bottleneck. GPU encoding is almost always faster than CPU encoding and it was a big deal when this was added to Premiere and AME a couple years ago. You can do a test by changing the encoding method under the Video tab to Software Encoding, but UHD footage takes a lot of horsepower to process and I'm going to guess that your GPU can process it faster than your 10-year-old CPU, but hey, give it a test on a short segment and see.

Video encoding is an incredibly complex topic, so simply looking at utilization value in the Task Manager isn't going to paint a whole picture. Someone else could post those same numbers and ask the opposite question of "why isn't my GPU being used more?" In fact, this is the dominant question that has been asked for the last 10 years with regard to rendering. When your CPU is being used more than your GPU and you're using Hardware Encoding, that means the frames being processed were likely not optimized to render on the GPU. Your transitions might not be GPU-accelerated, and the CPU still has to be used to decode other footage before it can be encoded to H.264. So what you're describing actually makes perfect sense, and as soon as AME could throw all of its processing onto the GPU, it did and encoding sped up.