Video preview in AE vs Video players

After Effects treats all source footage as if it's uncompressed for the purpose of compositing and special effects.

After Effects calculates every pixel on every layer every time, so building a video stream takes a little time. Once the preview is rendered, the cached preview should playback in real time.

Premiere Pro calculates every layer as a stream. I usually edit in 4K, and my Premiere Pro panel is set to 1/4 resolution. If I have only a couple of layers in Premiere Pro, playback is slightly slower than a media player but not as fast as a media player. If I have a lot of effects and layers, a real-time preview takes a while to prepare. Once the preview has cached, Premiere plays back the stream it has rendered in the background until you make a change in the timeline.

A media player decodes the codec, produces a stream from the file, does not report dropped frames, scales to the window size, and only read the original data. That involves a lot less calculating. Popular distribution codecs like H.264 (MP4) have such small file sizes because they throw away most of the data on two-thirds of the frames, average the color in blocks of 4 pixels, and predict the movement of detail between frames to deliver video stream that looks pretty good even though most of the original pixel data is gone. AE rebuilds every pixel in every frame and delivers 100% of the pixel data for every pixel every time. That's why previews take a few seconds to calculate, even if there are no effects and only one layer.

AE is getting a lot better, and so are all NLEs and compositing apps, but it will take a while before AE gives instant real-time previews.