Rendering doesn't use GPU

weelcome to After Effects. It mostly doesn't use GPU, and its multithreading is mostly just single-threaded rendering of frames, with one frame per thread, otherwide it would still almost be single core, no gpu. Most likely, none of your effects or edits are ones that have been have been updated for GPU use.
running these effects harder will make it even less likely that GPU rendering will work, as the other guy noted. As much as apple likes to suggest macs are a creator platform, the most popular media apps are old, and nearly impossible to optimize for their (or much anyone's) new devices without rewriting from scratch.

At some point any 3D acceleration has limits and creating a huge 3D scene is just one of those scenarios. In fact even just having 100 layers in 2D in a comp can do that because the GPU runs out of VRAM to load buffers for processing. I won't rule out any specific bugs biting your deriere, especially on a Mac, where this stuff is rather flaky, but overall everything seems normal within what you can expect from AE.

Mylenium

In each scene of the animation is one spotlight. The 3D layers have "metal" qualities, they partially reflect light and throw shadows. Most of the layers come from photoshop, only one video file is used.

Thank you for asking. I'm using only normal transformations and some effects inside AFX, no plugins. It's a stereoscopic 3D animation, each scene has got about 40-100 layers. Rendering never uses GPU, no matter if I render 2D or 3D, few or many layers.

And what's actualyl going on in the project? Using a single non-acclerated or incompatible effect or feature can cause this. unless you are more specific we really can't know.

Mylenium