Render speeds of h.264 compared to ProRes 422HQ

4k is about 4 times the amount of pixels compared to HD (UHD is exactly 4 times the amount, ‘real’ 4k is slightly more). So depending on how the comp structure is set up and what kind of effects are used, having a 4 times increase of render time is an actual possibility. It might not have anything to do with the codec.
But I’ll assume that in both cases you rendered at full res (4k) and same color depth and the only difference is the encoding part. In that case the rendering should be the same and the encoding to prores takes way more time than downscaling to HD ánd encoding to h.264.
That would indeed be strange and unexpected. Prores encoding should not take more than 4 times compared to h264. Unless you are rendering to a really slow external drive maybe.
As for the M1 ultra accelerators, as far as I know those are mainly decoders for prores, not encoders, but i’m not sure. H.264 and h.265 encoding is accelerated for sure, at least within some parameters (certain resolutions might not be accelerated)
The setting for using the decoders can be found in Preferences > Import>Enable hardware accelerated decoding.
Unfortunately for M1/M2 Macs it still advised to DISABLE this setting when using Prores as there is still a bug that can make prores files flicker greenish when enabled. Anyway, this concerns decoding, not encoding.
My experience with the M1 Ultra on macOS 12.6 is that prior to AE 23.1 the performance was really bad, with really long render times or even complete stalls for no apparent reason. From 23.1 on the rendering is back to expected speed and reliability, even when using Multi-frame rendering.