Playback cycles between using CPU and GPU. How do I fix this?

My problem is that when I'm editing a clip, that has effects, the playback of the clip in the timeline is sometimes done using my CPU and other times is done using my GPU. When the playback is done using the CPU it is laggy, choppy, and incredibly annoying to try and edit. When it is played back using my GPU it is clear as day and runs like a dream. I have determined this is happening by looking at the resources each of my components are using during playback. Without a fail whenever the playback is choppy my CPU is at 85 to 90% while the GPU sits idle at about 1 or 2%. When the playback is good, my GPU is running at about 50% with the cpu running anywhere between 20% and 50%. It is infuriating to edit like this because it's so random. I will play a clip and it will playback fine (using the GPU) and then in the very same session without changing anything I can rewind and go back, playing the same clip, and get a choppy mess because it is then using the CPU for the same task. It's incredibly irritating and makes editing a nightmare. It is also worth noting that I do have CUDA acceleration on and I run a GTX 1080 so the fact that it doesn't use it for every playback makes no sense to me. All my drivers and programs are up to date and nothing! Any help at all would be much appreciated, I'd just like to edit my footage with smooth playback!