Using JKL to scrub at over x2 speed eventually causes the video to bug out like it's stuttering or rubber banding

I find that when I press L at least 3 times or more to make the playback speed go faster, the video will play smoothly for a while but eventually it will start to stutter and flicker back and forth between the current playpoint frame and some other nearby one.

Sometimes the stuttering takes a while to start and other times less so. I normally have to hit K and then L again to make it go away, but it always returns. So when I’m trying to scrub through video, it’s a loop of
LLL → watch at the set speed → stuttering eventually begins → K → LLL → repeat
all the way through.

Not sure if it happens with J as I don’t scrub backwards as much.