Help with audio sync and timeline issues

I'm trying to animate a music video, but everytime I playback the audio, the video is synced at a different part of it. For example, If I know one line is said at frame 40, the next time I play it, it starts at frame 22. When I play the video, I can see the fps switching per frame, even though I have it set at 24. Some frames have as low as 9 fps. Does anyone know how to fix this?