AE Upper Limits

Break up the project. Anything that can be combined and pre-rendered should be done. Rotobrush, even after being frozen, can drag down a comp, so my standard workflow when using Rotobrush on a layer is to trim the footage, pre-compose moving all properties to the new comp and trim the new comp to the layer length, run Rotobrush, freeze, Pre-compose again and name the second Pre-comp "Roto" or something similar and Open the "Roto" comp then select Composition/Pre-render to render and replace the use of the nested comp in the entire project, then delete the "Roto" comp, so the AEP file size and stability is improved. If Rotobrush takes me longer than about 15 minutes, I always render and replace it.

I do the same thing with system-taxing effects like Trapcode Particular, Element 3D layers that take a long time to render, and Even EXR sequences that I'm using to extract data. I got the idea years ago when I observed how the folks at ILM and Pixar handled their projects. If there is anything in a project that is dragging the system to its knees, pre-render that section and move on.  Without seeing at least a flow chart of your project and the 300 + different parts that are dragging it down, it is pretty difficult to make intelligent suggestions. You could easily pre-compose a bunch of the extracted layers and render them because there is not much that can be done with an extracted ambient exclusion pass that can't be done with a rendered version of the same layer.

Plain and simple: No. AE can't be killed with hardware and even on the most basic "computer stuff" level you are simply breaking all limits. 300 image sequences is several gigabits of data transfer alone which will exhaust even the fastest SSDs and the PCI bus in general. The only option is to work smarter by using diferent file formats, splitting up the project into smaller segments and so on.

Mylenium

Sure there are limits, but then again 300 EXR sequences would bring even Nuke to its knees. See? Nothing unusual and there are no magic fixes here, even more so since AE has always been pretty terrible at dealing with EXRs in the first place. If it's an option, use more processing and file I/O friendly formats like TIFF, PSD, PNG etc. where possible, but even then you'll eventually hit a bottleneck somewhere.

Mylenium

I have the following specs

AMD Ryzen Threadripper 3990X 64-Core Processor 

Nvidia Tital RTX x2

100GB disc cache 

AE is given 208 of 256 GB RAM.