What's causing laggy playback

I've always had some level of issue with the playback of footage in Premiere Pro. Some things I record I have no problem with the playback and others I can't play more than a couple seconds of seconds without everything freezing. 

This makes making things like keyframes a nightmare.

The most recent example is in the footage I recorded of the game Subnautica. The game has a day/night cycle and everytime it is night in the footage the playback is extremely laggy/unplayable. 

I’ve tried just about everything in terms of fixes within premiere pro like clearing caches, etc…

I record all my footage in MKV and then convert to MP4 through OBS. 

I also have what I would describe as a Higher end pc so I don’t think its a hardware issue either. 

Wanted to post here and see if anyone had any idea why this happens to me and see if there are any potential solutions.

Thanks!