Mixing Frame Rates of Micro Variations (and larger ones)

Report · Feb 16, 2023

Greetings video nerds!
I think I need to get clarity on mixing frame rates inside a project sequence/timeline before I continue working on my showreel type of projects. Here's a real-life use I've got right now:

Commercial Demo Reel.

Goal here is to mix footage from different commercials I worked on
Source audio has been removed from footage. Using only picture, and I will add a separate music track.
Most were shot at 24 frames per second.

ISSUE: The frame rate shown in the project window varies – I have of course 23.976, but I also have 23.93, 23.98, 24.00, and 24.04 showing in the frame rate column. These are clearly intended to be 24 frames per second so…

QUESTION: Why are they showing these fps discrepancies? Why are they not all exactly 23.976?

QUESTION: Can I safely mix these in the 23.976 timeline, even if I get a warning that the little variations don’t match the sequence settings?

QUESTION: Taking this a bit further, I also have some source footage that was shot at 30 frames per second that I want to throw in there. Most of it is 29.97, but again, some are 29.92, 29.95 30.01, 30.02, etc. How do I handle these? Same way or differently?

There seems to be a lot of misconceptions around the Interpret Footage feature. I’ve seen some people say do Interpret Footage to all source material to create a congruent frame rate to work with in a sequence, and to avoid dropped frames at all cost, but as a 2nd step make sure to also do a clip duration of a certain % to bring it back up to the actual speed it should be. I also haven’t been able to find a YouTube video that explains what to do when the frame rates have micro variance (23.976, 23.93, etc)

BONUS QUESTION: If I wanted to keep the source audio of this mixed fps footage, would that affect any of the recommendations given here? If so, how?

If someone could give me definite answers to my questions here that would be so wonderful!

Report · Feb 16, 2023

First, any clip showing a non-standard time ... those odd .04 like things? Are clearly VFR ... variable framerate ... meaning the capture device was set to a certain FR but continuously changed actual recorded FR during the clip depending on motion/detail 'need'.

I would recommend using the free ShutterEncoder utility app to convert those to CFR ... constant framerate. Then use the files created by SE to replace those in your project. And at the same time, you can make the FR more consistent as a bonus.

For standard video media, Pr tends to work pretty well with mixed FR sequences. So ... make a test sequence, either by creating a new sequence from settings you set or by selecting a clip with the right framesize/rate for your project.

Then drop several clips of different FR on it. Does it work correctly?

Neil

Report · Feb 22, 2023

Hi Neil, thanks so much for taking the time to answer this post and for introducing me to ShutterEncoder. If I understand correctly this conforming of VFR clips to CFR is something that cannot be achieved using Adobe's included Media Encoder? This is something that absolutely requires a 3rd party solution?

Also, such "conforming" from VFR to CFR - will that change any of these source clips' duration, even when dealing with such small variations in frame rate? If so, how long before it starts to show an "out of sync" problem? For example, I have another project - a music video - where I shot a few angles using my phone while also capturing clean audio of the musical performance separately. Will I run into sync issues if I try to sync the audio track I captured to source VFR footage that has been conformed to CFR? Since the variations of frame rates are so small, will it eventually lag (or get ahead) of the audio track? I would think that the duration of the video would have to match the duration of the audio but ...You're the expert. 😉

Finally, what did you mean by "For standard video media"?

Report · Feb 22, 2023

Neither Premiere nor MediaEncoder have the process to convert VFR to CFR, as they simply weren't built to handle non-standard video files.

So you have to use an outside app. ShutterEncoder, like Handbrake, use the ffmpeg open-source coding to work with long-GOP H.264/5 media. They are very good, free, and useful utilities. There are for instance more options for dealing with file sizes in them than in Premiere or Resolve for that matter when encoding to H.264 type files.

So there are many pro users that will export a say ProRes 422 or 444 "master" file of the project. And then use ffmpeg, Handbrake, or ShutterEncoder to make the H.264 deliverables for b-cast/streaming/whatever.

ffmpeg requires the user to know and use the correct code phrases. Handbrake is a UI built around ffmpeg's processing that is a bit long in the tooth. ShutterEncoder uses ffmpeg as the processor but is a much modernized and more understandable UI.

Does this change file total length/duration?

No, shouldn't make any change whatsoever. VFR media has a constant speed audio track, with constantly varying numbers of frames per second. Converting to CFR simply means creating actual frames at the set frame rate. Doesn't mess with file duration nor take things out of sync between audio & video.

So rather than the number of frames per second changing constantly throughout the file, there will be a set framerate with the correct number of frames per second as a continuous playback.

By "standard video media" I meant normal, fixed-framerate CFR media. Which can have different speeeds it was recorded at, but each is CFR within that set framerate. Such as 23.976, 24, 29.97, 30, whatever.

Neil

Report · Feb 22, 2023

Ahhhh ok! That makes sense!

Now, honestly, years ago, I likely ended up making project sequences that mixed CFRs AND VFRs and my eye never picked up on anything problematic.. and I'm guessing alot of people don't know that VFRs should be conformed first, at least as a best practice. So it wasn't that I noticed something was "wrong" that brought me here - it was just that I had noticed the discrepancies in fps between my source clips in the project panel and couldn't understand why, and how to handle it. I was paranoid it would create dropped frames, which seem to have a really bad rap on the internet.

So I guess my next question to fully wrap my head around this... if one does not bother comforming all VFR footage to a consistent frame rate as you suggest, what would be the "symptoms"? How would this negatively impact the final export? To me, the variations seem so small that I wasn't even sure if mixing varying VFRs would be noticeable or what exactly I would see that would indicate there's an issue. Can you expand on this a little please?

Report · Feb 22, 2023

Premiere can sort of mostly work somewhat with VFR clips.

Note all the qualifiers there? Yep.

In other words, it works till it doesn't. Because of some effects or a concatenation of editing steps or whatnot.

Any time changing is problematic off the bat, both speed ramps and Warp.

Plus Premiere can only export CFR files. So it essentially has to guesstimate the frames.

Sometimes it does OK. And sometimes you'll look at the export and see oddness happening in a frame here or there.

So it's a gamble with your time, innit? As it might work or you might have to rebuild it.

As you can batch convert in either Hb or Se, while you're working, sleeping, or at lunch, why take the gamble?

Or use your laptop for the conversion while working on your desktop.

Neil

Report · Feb 22, 2023

Thanks Neil,

So then Interpret Footage does not do what I thought it did.

Before starting this thread, I had done Interpret Footage on all my clips that were fluctuating slightly from the "standard" frame rate values. And I thought that was the right thing to do, without knowing exactly what I was doing to my clips (behind the scenes). It seemed like a good idea and the frame rates shown in the Project panel were now all congruent. But it sounds like it just seemed to do the same as what ShutterEncoder is doing.

I was even about to do it on my source footage that's 59.94, which I want to use at regular speed in my 23.976 or 29.97 timelines.

So if they don't do the same thing "under the hood", when would I want to specifically use Interpret Footage?

Report · Feb 22, 2023

Interpret Footage has a couple functions. You're talking about only clip speed here.

The devs have told me the the Interpret Footage "apply X framerate" is meant primarily for changes either involving cadence changes (a rabbit hole of its own) or when you want Premiere to treat it as a particular speed, different than say when simply dropping it on a timeline.

It has nothing to do with variable framerate changes. Nor general speedramps either.

Neil

Report · Feb 22, 2023

When I look at my list of footage in Premiere, is it safe to assume that everything showing 23.976, 24.00, 29.97, 30.00 and 59.94 are all constant frame rates? (If it is, then I won't have to open the properties box of every clip I imported.)

Alternately, I was going to ask if any footage that is NOT these exact values means it must be variable frame rate and should be conformed into a constant frame rate first? As per the logic of your first reply "any clip showing a non-standard time ... those odd .04 like things? Are clearly VFR .."

However, I just looked at the properties of a few clips that are 23.98 (which by my logic and according to your answer should have been VFR) but when I looked at its properties inside Premiere, they did NOT indicate "Variable frame rate detected".

So I'm a bit confused now, because I thought odd numbers like that meant they were variable frame rate, and an easy way to tell if something needs to be conformed to something standard but this breaks that logic. And because of that, I don't know if I should still conform this one to 23.976 in ShutterEncoder or just drop it in the timeline since it is NOT variable frame rate.

Any advice? I'm sorry if it's taking me a bit to understand.

Report · Feb 22, 2023

Go with the Properties listing of VFR ... that would be more accurate.

Remember, the VFR "listed" framerate is a fudged figure, not an absolute. As in a clip that is "23.98" in VFR may have bits down to 22.44 or up to 24.6, just to list a couple variations I've seen in checking files.

That's the whole thing ... that framerate isn't constant. It varies.

Neil

Report · Feb 22, 2023

"Go with the Properties listing of VFR ... that would be more accurate."

What does this mean? Drop the 23.98 mentioned here inside the timeline as is ... and trust what the properties panel says instead of just relying on the fps shown by Premiere because there's no hard rules that a 29.97 or 23.976 (for example) are always fixed/constant?

Report · Feb 22, 2023

If the Properties panel or MediaInfo (a great free utility for data about the files) say it's VFR, it's VFR.

Neil

Report · Feb 22, 2023

i think i may not have expressed myself correctly here...

My point was that a few 23.98 clips appear to be CFR (not VFR) according to the properties, which is contrary to expectations if just looking at the number shown in Premiere. (this one isn't smartphone footage but TV commercial I got from a client years ago.)

Following your suggestion, I downloaded MediaInfo to compare and there is indeed a discrepancy between the frame rate reported by Premiere vs the one reported by MediaInfo. Sucks that we can't trust the numbers shown by Premiere.... It does however seem to accurately report whether something is Variable frame rate or not in the properties.

Premiere vs MediaInfo - FPS discrepancy

Moving forward to establish my workflow, maybe I should just trust that whatever I was sent from a production house is CFR, and whatever raw footage I shot with a smartphone or camcorder is VFR? Otherwise I feel I would need to investigate the properties of every clip before I start assembling - is that normal? (I'm a newbie)

I think that the screenshot illustrates an example of a situation where I might want to use Interpret Footage so that I can tell Premiere that it should be interpreting/reading this footage as 23.976 and not 23.98. This will also have the benefit of reporting the accurate frame rate (as reported by MediaInfo) inside the Project panel. Would you agree?

Report · Feb 22, 2023

Huh ... Premiere rounded the 976 to 98, that's something I've not seen before. Weird!

I'd tend to have the same assumptions. Pro houses will send CFR, most camcorders will also do CFR. Most phones will do VFR, and some screen capture apps like OBS default to VFR.

But I went into settings on OBS and set that to CFR and to mov.

Neil

Report · Feb 23, 2023

Well, the whole 23.98 is weird to me. Is that considered a commonly used frame rate?

Next to this thread was a forum related one https://community.adobe.com/t5/premiere-pro-discussions/mixing-frame-rates-and-selecting-a-new-frame... where the client requests 23.98.

Why would someone request 23.98? I thought 23.976 and 24.00 were the standards when referring to 24fps?

Report · Feb 23, 2023

In years of video post & production ... I've never seen Premiere list a 23.976 file as 23.98. So to me, that is an oddity par excellance ... well, and someone asking for 23.98!

Now that you've got MediaInfo, you can quickly check the actual framerate ... and see things like the full "30,000/1,001" thing to show precise timing.

And of course, there's also Drop-Frame and Non-Drop Frame ... just to get even murkier.

@Warren Heaton is one of the folks 'here' that has the best knowledge of working framerates. Experienced b-cast person that he is.

Neil

Report · Feb 23, 2023

There's p24 and p23.976 (p = progressive scan).

"23.98" is just another way of referring to 23.976.

If someone writes "p23.98", then they are asking for p23.976. Or if software indicates "23.98", then it's indicating 23.976.

If manually entering the frame rate, let's say in the After Effects Composition Settings dialog box, it's really important to enter 23.976.

If making a written note of a frame rate to be used for a project, let's say in an email, write "23.98" or "23.976" (but know that it's really 23.976).

I would avoid rounding 23.976 or 23.98 up to 24 as that just creates confusion between p24 and p23.976 (so avoid rounding up to a whole number when talking frame rates).

Quick summary of frame rate history:

24 frames per second comes from an international standard established for motion picture with sound (before video and when moving picture was analog).
Analog video establishes 30 and 25.
Around 1969, 29.97 resolves the phase shift of blue in analog video transmission and timing 30fps content accurately to the hour.
23.976 (also referred to as 23.98) accommodates reversing 3:2 pulldown which wasn't a thing before motion picture became digital.

If you're really looking for a small headache, think about how i59.94, i29.97, i60, and i30 are all valid ways to refer to interlaced drop-frame video with 60 half-vertical resolution images per second.

Then think about progressive segmented frames or how the term "progressive scan" is still so widely used even though CRT displays have mostly been replaced with flat panel displays that have a refresh rate rather than a scan method.

Report · Feb 23, 2023

Warren,

I don't know how many times I've either read through or watched a detailed discussion of drop-frame versus non-drop-frame ... my brain still goes ... huh? ...

And then someone breaks into a discussion of how the pull-down cadence and DF/NDF works if X but not Y, but if Z you need to do Q instead ... hoo boy.

Brain freeze ...

Report · Feb 23, 2023

You want to treat your demo reel workflow like a documentary workflow.

What will the settings be when you have a full-resolution, stand-alone edited file? The answer to that informs how you conform source footage settings and configure your Timeline settings.

Assuming that your demo reel will be uploaded to YouTube, Vimeo, or FrameIO (in case you haven't looked into it yet, a FrameIO Presentation Link is a great way to share a demo reel with prospective clients via email, text message, Slack message, Discord, etc. and an Adobe level FrameIO plan is included with our Creative Cloud subscription), then ProRes 422 LT 2160p24 or 1080p24 (true 24 or 23.976) is a good option unless the majority of the source footage is i29,97.

Report · Feb 23, 2023

QUESTION: Why are they showing these fps discrepancies? Why are they not all exactly 23.976?

Answer: It sounds like your source footage is a mix of delivery formats where frame rates are not locked in as they are with edit formats (or "mezzanine" formats or "intermediate" formats). Sometimes there are irregular frames that throw what should show as 23.976 or 29.97 off. If the various frame rates were showing with camera originals or a mezzanine format, I'd worry. If they're showing with delivery files (ripped DVDs, downloaded YouTube, screen-recorded playback, H264 MP4s, and the like), it is what it is.

QUESTION: Can I safely mix these in the 23.976 timeline, even if I get a warning that the little variations don’t match the sequence settings?

Answer: Maybe. For a demo reel, you're probably fine leaving the source as is. If this were a paid gig, I would transcode any footage showing an irregular frame rate to ProRes in After Effects, locking in 23.976.

QUESTION: Taking this a bit further, I also have some source footage that was shot at 30 frames per second that I want to throw in there. Most of it is 29.97, but again, some are 29.92, 29.95 30.01, 30.02, etc. How do I handle these? Same way or differently?

Answer: For a demo reel, you might be fine leaving the source as is and testing which Time Interpolation method (Frame Sampling, Frame Blending, or Optical Flow) in the Clip Speed/Duration dialog box for 29.97 sources works best in your 23.976 Timeline. Optical Flow usually works best, but it varies based on the source footage. Avoid assuming a different frame rate in the Interpreting Footage options for source clips unless the source is without sound. If this were a paid gig, I would check if the 29.97 source has 3:2 pulldown and then pull that up to 23.976 in After Effects and render it to ProRes 422 LT 23.976. For interlaced 29.97 without 3:2 pulldown, I would Separate Fields in After Effects and then use Timewarp in a 23.976 Comp to render it to ProRes 422 LT p23.976. If not totally happy with those results, I'd run them through Apple Compressor (which inherited solid Frame Rate Remastering from Shake). It also would be worth running a clip through Topaz Labs Video Enhance AI to get from i29.97 to p23.976. Its upconversion is excellent. I haven't tested its frame rate remastering.

BONUS QUESTION: If I wanted to keep the source audio of this mixed fps footage, would that affect any of the recommendations given here? If so, how?

Answer: For a demo reel, add it to the timeline and see what happens. If taking the route of transcoding to ProRes to conform to 23.976, leave the source audio with the file to hold sync. If a change in pitch is noticed, adjust for that during the audio mix.

Report · Dec 16, 2023

Hi Warren - picking up from a few months ago! Thanks for taking the time to really go into details with this. I'll admit it was a little bit over my head because I'm not a video "expert" and I mostly just make demo reels for my own showcase,, to post on YouTube/Vimeo in 1080p. I'm not hired by any company to produce award-winning reels... at least not yet lol. But I do obviously want my reels to look as best as possible, and more importantly, I just want to make sure I avoid any pitfalls I may not be aware of. Most tutorials I've watched just drop in their footage and let Premiere do its magic. And their exports look fine enough for me!

I would like to ask though... you mention conforming (or transcoding).... not sure if those terms are interechangeable and mean the same thing... but you described a specific process using After Effects. Could I achieve the same simply using ShutterEncoder or Handbrake? Not sure if what Neil had suggested is the same thing you're suggesting - just using a more simple software to achieve it.

Report · Dec 16, 2023

"Transcoding" is inputting one format/codec, and exporting a different format/codec.

For media with the truly odd framerates of say 27.82 ... anything not a "normal" 23.976, 29.97, or 59.94, it's probably "variable framerate" ... screen capture apps and many phones and some GoPro/drone media are of this ilk. Audio is constant, but the framerates constantly shift as the device only records the bare number of frames it needs to keep detail fairly ok.

None of the Adobe apps fully correct that to constant framerate that I know of. So that's where using Handbrake or ShutterEncoder is useful, as, if you set the controls to do so, they can correct this and output straight media. I tend to like S-E, but ... they recently moved the "set to constant framerate" option to deep in their preferences, so you have to hunt for it.

"Conforming" is the process of taking say the clips of a sequence from one app and getting them into another, with any scaling changes, speed changes or ramps, and audio files correctly reproduced.

This is done of course if you send your project to someone to color/grade it in say Resolve or Baselight. You send the media, an "XML" and/or "EDL" file, and a low-bitrate H.264 of the sequence. They use the XML and EDL files to get most of the clips laid out on the timeline, put the H.264 file on V2, and go through and check that all clips are correct in to out, cuts, scale & speed changes.

Conforming is also sometimes used to refer to cadence corrections ... that 3:2 "pulldown" that Warren mentioned. You should look that up, it's interesting if you don't need it but absolutely necessary to know for when you do!

And no, S-E and Handbrake can't do pull-downs, but Ae has one of the best utility functions for that around.

Pull-down is needed especially for some interlaced to progressive, and progressive to interlace, with certain framerate differnces. If you are prepping for the web, probably not much of an issue. Premiere seems to handle cadence changes pretty decently for most usage. You might notice a repeated frame, or one might appear to be skipped, on occasion. But in general, interlaced to progressive works ok.

You cannot give that to any broadcast facility though! Many broadcast facilities still require interlaced, and in the US in 29.97, and ... that must be perfectly retimed and interlaced. Ergo, using Ae after you learn what it is you're doing.

So ... S-E and Handbrake do great for converting VFR to CFR. For transcoding in general, Premiere and MediaEncoder can do say H.264/mp4 that is already CFR to say ProRes quite handily. And can convert progressive to interlaced.

But ... if you need to get proper broadcast spec 29.97 interlaced, you may need Ae and setting pulldown or up corretly.

Have I stirred the mud adequately? lol

Report · Dec 16, 2023

Yes you have!

OK simple question for a specific use. I have a 20 minute performance captured by an Android smartphone that shows a frame rate of 29.77. To be dropped in a Premiere sequence of 29.97 and intended use is web only. Should I bother running this file thru ShutterEncoder? I don't think I notice anything off when playing it back in Premiere. And if you do advise I run it thru SE, which menu option would I be choosing? I'm a little overwhelmed by the very long menu of functions... would I be looking for something under editing codecs? Or output codecs? I then see under Advanced Features something called "Conform by" -- Blending (default mode selected) > 29.97. Which I'm assuming is the only value we are looking to edit to transcode this from VFR to CFR?

Again, is this something I need to do since I'm not broadcasting this or noticing anything weird during playback?

Report · Dec 16, 2023

Do a test export from Premiere ... if it looks fine after export, you're probably ok.

In ShutterEncoder (S-E) ... you have to go into the Settings or Preferences options ... hard to find, actually, but there's a little icon in the far upper left that you click on to get to the setup options.

Once there, scroll down until you see convert to standard framerate. Check that option.

Now ... go back to the main S-E screen, and when you set a transcode to operation, to any other codec, it will also convert that file to CFR. Make sure you set the framerate you want, of course!

In HandBrake, there's a little circle you check for "CFR"

Report · Dec 16, 2023

OK thanks I found it as "Set framerate mode : cfr".