Getting AVC VFR to CFR = lip sync challenges

[Peter25587330m3rd]

I have a clip of 4K AVC footage from my Google Pixel, with MediaInfo reporting VFR 24.534. Premiere hates it (playback and scrubbing is jittery and hangs constantly). A discussion on this forum led to "the problem is the VFR footage - change it to CFR and a codec meant for editing".

So I'm trying to get the footage to ProRes422 and CFR, however I have lip sync issues. The audio keeps up in general, however there are occassional, significant inconsistencies with the lips matching the audio. It's not usable.

I use Shutter Encoder to change the clip to ProRes 422. I've posted on the Shutter Encoder subreddit re: various settings (I included that below, just in case someone here knows the software), but figured I'd ask some general questions here.

1. I'm assuming the lip sync issue stems from taking VFR and forcing it to CFR. Is that true?
2. Is there a solution to get it to CFR other than re-encode?
3. Since the VFR has the best audio sync, should I look for some other solution for editing the VFR footage? Like what?

I appreciate any insight.

Shutter Encoder stuff

Here are the trials I've done, adding my clip and selecting ProRes 422 as the codec:

- keep all settings at their default = resultant footage is still VFR + lip sync issues

- in Settings, "Set framerate mode" to CFR = footage is CFR, 30fps + lip sync issues

- in Settings, "Set framerate mode" to auto; in Advanced, set Conform By, blending, to 25fps = footage is CFR, 25fps + lip sync issues. I've also tried using Conform By with 24fps and 30fps = same results.

[Ann Bens]

Is your sequence settings set to 24,00 and not 24.534.

[Peter25587330m3rd]

I'm not even getting it into APP yet. I'm re-encoding the clip and then watching it in VLC.

[MyerPj]

What's the framerate set to in your Pixel? If it's 30, then I think your 2nd test is valid, but if its 24, then, I would try again, leave the preference "Set framerate mode" to CFR always. Never think about it again! 🙂

Try in Advanced to Conform by... and 24.

A mediainfo in treeview of your two files would be great.

From my Pixel Pro 8 I get of course 😞 VFR, when I transcode it I often use ProRes 'Proxy' as that most closely matches the bitrate coming from the Pixel.

[Peter25587330m3rd]

I can choose 30 or 60, but nowhere can I choose whether its CFR or VFR. So it's set to 30, and I'm getting approx 24fps VFR.

[MyerPj]

I just tried an 8.06 pixel file into SE to ProRes Proxy and the lip sync seems good. How long is your clip?

Original

ProRes Proxy

[Peter25587330m3rd]

Would you mind reviewing steps you took to generate the ProRes proxy? Did you use AME? Any particular settings other than default?

[MyerPj]

BTW: The ProRes proxy is about 4 times the bit rate of the Pixel file, so I think that's enough.

[MyerPj]

Just like you did, using Shutter Encoder but change this drop-down to Proxy (or whatever else you'd like) for the ProRes 'flavor'. 🙂

[Warren Heaton]

@Peter25587330m3rd 

Regarding frame rate, you want to transcode to what the application MediaInfo (shown in a different reply) shows as the "Real Frame Rate".  You don't need MediaInfo to find this as it's what ShutterEncoder will use automatically, but it can help to know what it is.

You can also find this in Premeire Pro by right-clicking the Clip in the Project tab or Bin tab and choosing Media File Properties.

If after transcoding at the "real frame rate" frame rate you are still having a sync issue, something else may be going on.

Have you had time to try other clips recorded with the same device?  Hopefully, it's just this one and you can split the audio track and nudge the resulting audio segments every so often to maintain lip sync as needed. 

Something else to check is the sustained data transfer rate that you are getting from your storage media.  I like to use Blackmagic Design Speed Test for this.

[MyerPj]

It's assumed you've got CFR set in the Shutter Encoder prefs. 

Now you need to post a screenshot of MediaInfo in Tree view, similar to the one I posted above.

You can use the free MediaInfo program to get info on your clips. Get the program from the link below and using the 'Tree' view post a screenshot of your clip info: https://mediaarea.net/en/MediaInfo/Download

[Peter25587330m3rd]

To be clear, the audio sync issue can't be fixed by moving the audio back or forth. Sound is generally happening when it's supposed be happening. The video is a vocal performance, and while the original VFR footage has perfect lip sync - all conversions/encodes I've done with this footage ends up with the sound not matching with the lips (while the audio and footage remain generally in sync, e.g., the song stops and people applaud right when it happens in the original footage.

It's almost as if the video is speeding up and slowing down in very fine increments constantly. Sometimes the sound is perfectly in sync, then it drifts out of sync for a few syllables (and honestly the footage does look slow-motion-ish for very quick moments) and then it's back in sync again for another few moments.

I've now tried keeping the footage and Shutter Encoder on my C: drive, which is solid state connected to the MOBO. Same results, whether I use ProRes 422 or ProRes Proxy.

Below is the Tree view from MediaInfo, of my original footage - anything weird?: