Dropping Frames with very little hardware uttilization

My sequence consists of a single low quality video with no effects. During playback CPU usage is under 4% and only about 2gb of RAM is being used by premeire (8gb of 32gb across my entire system). Nothing else is being used more than 1%. Despite this, my playback is incredibly choppy and I am dropping about 2 frames/second on the indicator. What is going on?