VFR to CFR, Optical Flow stutter

[avidthrows]

I recently screencapped some footage of an interview I did over Skype. Skype is vfr and vbr, but the method I used to record, OBS, captures constant framerate only, and therefore there are plenty of duplicate frames in my footage.

What I want to do is remove the duplicate frames then interpolate between them using Optical Flow to reduce stuttering in my footage.

Is there a way to easily do this in premiere?  If I apply Optical Flow to the clip AS-IS the footage will stutter the same amount it already does, since Optical Flow interpolates between the same duplicate frames.

This would also be useful when editing video-game gameplay footage that is already converted from VFR to CFR. (As is the case when doing any sort of livestream video editing) Especially useful too when doing In-Engine game development demo footage that pushes the hardware to its frame-stuttering limits.

[jasontcox]

Optical Flow is a bit of an odd duck in the Premiere timeline. When you say that "If I apply Optical Flow to the clip AS-IS the footage will stutter the same amount it already does, since Optical Flow interpolates between the same duplicate frames" was this in the timeline or the export? Because Optical Flow is one of the few "effects" in Premiere that does not playback properly in the timeline/Program Monitor unless you render it/create a Preview File.

[avidthrows]

I know that I need to Render In To Out with Optical Flow. That's not a helpful response, I don't think you understand the problem here.

[Ann Bens]

Optical flow will only work on clips with speed changes or time remapping.

[avidthrows]

Incorrect. If you have a 30fps sequence and change the sequence settings to 60fps it can interpolate frames and maintain the duration-time of the sequence. I've used this feature before.

That's not within the scope of this question though.

[chrisw44157881]

right click interpret footage fps on video in project view

or

In the Project panel, right-click a sequence, and select Sequence Settings.

[chrisw44157881]

premiere uses frame duplicates when the interpreted fps doesn't match the sequence settings. change one or the other.

[avidthrows]

exactly how would I go about doing that with vfr footage that's already been converted to cfr without frame interpolation?

[avidthrows]

So nobody has a workable solution for this?

Is there even a non-adobe workflow for (VFR- > CFR ) -> VFR -> (CFR-Interpolated)?

(._. ) I realize how stupid ^that looks but sometimes we have to work within the constraints given. Footage we have vs footage we want.

[Carlos Ziade]

What's the frame rate of the 'OBS' recording ?

What's the frame rate of the sequence you are working on ?

What's gonna be the frame rate you will use for export ?

These are essentials to know to be able to give more accurate answers

[avidthrows]

The fact that you're asking this tells me that you don't know what VFR means.

[Carlos Ziade]

the fact is that we are trying to help you here

[avidthrows]

The footage was originally Variable-Frame-Rate, which was then converted to 60fps. THAT is why it's stuttering when I try to use optical flow to convert it to 120fps.

When you convert VFR (variable frame rate) to CFR (constant frame rate) you get frame duplication, and Optical Flow is interpolating between those duplicated frames, effectively generating the exact same frame between any frames that are duplicates.

My question is how can I convert-back the footage to VFR inside of Premiere, and after doing-so, apply Optical-Flow to it to smooth out the framerate fluctuations and frame drops native to streamed-video. And if that's not possible inside of Premiere, if there are external tools that will allow me to do this. I do not have access to the raw VFR footage, only post-converted. This was all in the original question.

Googling for a solution leads me to a plethora of VFR to CFR solutions, but that's the exact opposite of what I'm looking for. I've tried using ffmpeg with the decimate filter to export vfr mp4 in the command line, but it removes all the time-values for those frames, which results in a video that instantly goes out-of-sync with the audio and fluctuates heavily in speed at every previous framerate-transition.

[avidthrows]

Even if I right click the footage, Modify>Interpret-Footage, nowhere in that dialogue box is there an option to interpret it as VFR, and if there was, it's technically not vfr video; it's been converted. Premiere automatically assumed, correctly, that it was 60fps. Because it is. Technically.

So modifying the source clip this way does nothing for me. I've tried. Problem still exists, and I've explained why it does already.

[Carlos Ziade]

I was checking stuff related to OBS, the method you used to capture the video,

I never used this before, but I think there is a File - Remux Recordings in the main OBS studio window ?

I don't believe you can change from constant to variable frame rate in Premiere Pro for now

[Stan Jones]

I assume you are using PR 2018, since VFR wasn't supported before 2018.x.

Have you tried using the VFR version and using modify there?

No idea what will work....

[Carlos Ziade]

You have to try to do that in third parties or OBS studio as I mentioned earlier, Optical Flow will not

do the interpolation as you want it, as Ann said earlier, Optical Flow is mainly used on speed changes

and time remapping. That is why I was stressing before to know what was the OBS frame rate recording

and to what frame rate are you going to export, you are going from a lower frame rate to a higher frame rate

for a VFR footage that was transformed to CFR footage.

So now you have to focus on remuxing your footage back to VFR throughthird parties,

f that is possible through OBS as I hinted earlier. To have features that would enable you

to encode/decode from VFR to CFR and from CFR to VFR we need to submit that in

the features request, and I will vote for you, this is important for all people going your workflow.

@Stan Jones the user does not have access to his VFR footage, only the CFR captured by OBS