Usually, rendering footage or simple motion graphics at 8k (7680x4320) works just fine. No issues in terms of functionality.
Lately I've been working on a project involving Video Copilot's "VC Orb" plug in, with an 8k texture applied to it. I am able to seamlessly move around my timeline in "quarter" resolution but once I set it to "Full", it never updates my preview and After Effects inevitably loses all functionality, forcing me to restart the application. Also, when I set it to render as a ProRes 444 in AE or AME, it begins the render but no progress is shown on the render bar and the elapsed time stays at "0 seconds" (the loading bar showing how much of the video has been rendered doesn't do anything).
My project settings are set to GPU accelerated encoding so I'm sure it's utilizing my GPU which is a good thing.
In the NVidia control panel, I went to manage settings and for After Effects, made it so eachsetting utilizes my graphics card. Also changed my processor to my GPU in the PhysX settings.
As a test, I began the render last night and came back to it this morning and it had not moved at all. What's weird is AE doesn't freeze or crash during this, I can pause/stop the render and freely click around, it just doesnt even try to begin actually rendering the shot. very weird.
Try restricting the video memory of the graphics card and possibly enforce an older version of openGL to improve compatibility. Could well be that the OpenGL client in the VC plug-in is simply blackholing (trying to address a memory area that doesn't exist) or does not get a correct return command from the graphics driver.