Variable Frame Rate Video is extremely jittery and stuttery.

I have a PVR that I use to record gameplay videos. It does indeed record in a variable frame rate, but it never posed a problem for me since my gameplay videos were rather short, so the audio never fell out of sync that much; and if it did, I could just nudge the audio a frame or two. I never had the necessity to use HandBrake or anything like that. In the new update, which added official support for VFR video, all of my previously smooth footage has become extremely jittery and way out of sync on both the smooth video and audio sync modes. I have no choice but to use HandBrake for all of my new footage.

Before you ask: Yes, I did clear my cache; and yes, I did go to the cache files and manually delete everything.

This is happening not just to my older footage, but my newly recorded footage as well.

In the preview window and in the final exported clip, the footage is way way out of sync, keeps skipping around, and freezes every once and a while.

Does anyone else have this problem? Is there a way I can fix this without the use of HandBrake?

EDIT:

     I just restarted Premiere and the audio and video seem fine, but then once I start scrubbing around, that triggers the sync and jitter to start.