Upscale 8mm digital converted film to 1920x1080 HD

[John Ratard]

I would like to include 8mm film @ 30fps digital conversion 720x480 type 2 AVI (dvsd) from 1950-1960 into a recent 1920x1810 60p AVCHD master Premier Pro project. I was hoping to use After Effects to upscale the 720x480 to 1920x1810 if possible.

I have reviewed forum instructions for a 720x480 60i DV (non square pixels) upscale to 960x720 60p (square pixels) as follows:

1. Import the DV master into After Effects CC
2. Right click the file in the project bin, go to Interpret footage, then Main
3. Set Conform frame rate to 59.94, hit okay
4. Drag this movie to the new composition button
5. Select the composition in the project bin, hit Command+K to open the comp properties
6. Set the frame size to 960x720
7. Set the duration to double the original
8. Set pixel aspect ratio to square, hit okay
9. Select the video layer in the new comp, then go to the Layer menu, select Time, then Time Stretch and enter 200, then hit okay, the number should match the frame duration you just got from doubling the original
10. Now, add the Detail preserving upscale effect
11. Click fit width
12. Export

As I am new to upscaling with this more ambitious project with lots to learn, how should  I proceed?

[Dave_LaRonde]

What's the original frame rate of the film before it was converted to digital?

Is it 4x3 video?

Why screw with the 960x720 comp when your target resolution's 1920x1080?

No matter what you do, even pushing Detail Preserving Upscale as far as it can go, this stuff's going to look really bad.

And I don't understand why they're having you fart around with the frame rate.

[John Ratard]

The film frame rate was 8fps using a spring wound Canon video camera with revolving turret.

I used GSpot to determine the avi 720x480 4x3, frame rate video format.  A telecine like capture arrangement included Cinecap software with a WorkPrinter-XP, a converted projector at 6 fps, a mirror and lens, and a converted mouse. The software captures the 6 fps images and converts to the avi file. A DVD of the film shows a reasonable frame rate.

The converted film is already of poor quality due to storage in high humidity of Vanuatu. By upscaling I was hoping to minimize the degradation.  Perhaps I do not need to be concerned with frame rate since the export will be imported into the AVCHD Hires setup in Premier Pro and converted there.

[thinkshock · Accepted solution]

I'm going to assume that your target is actually 1920x1080 and that 1920x1810 is just an error because I'm not aware of that resolution.

The moment you create a 1920x1080 composition it will automatically be a square pixel format and your source footage will convert to square pixel once you place it in the composition.

You don't need to do the intermediary step of creating a 960x720 composition. You can use "Detail Preserving Upscale" inside of the 1920x1080 composition and it will get you there in one step. I believe it's a 225% scale, but the effect will allow you to automatically scale to the height of the composition.

The end result should be a 1440x1080 image inside of the 1920x1080 composition with black bars on the side.

If you choose to the fill the whole frame you will either have to stretch the image horizontally making everyone look a little fat or scale the image up even further so that the top and bottom of the original image are cropped out but you will maintain the original aspect ratio. I personally don't like scaled up or stretched images, but it appears to be a common choice these days.

You do not have to change the frame rate of the footage unless your really, really want to. 60i and 60p are the same frame rate and realistically your footage originated as a 24fps source that has probably been through a telecine process to 29.97 fps.

All the frame rate variations can be very confusing and complicated but as long as your source footage doesn't have sync sound it's going to be hard to mess things up. However, if there's actually sound attached to the footage, changing frame rates has the potential to make things go south pretty fast. Without getting too technical an old rule of thumb is to keep decimal frame rates together (23.976, 29.97, 59.94) and whole number frame rates together (24p, 30i, 60i, 60p, etc.) It's difficult to explain working with different frame rates, which tends to require a lot of working experience (and some horrible out of sync deliveries) to really appreciate the issue.

There is possibly one other thing you will have to do. If your footage looks jagged or seems to have kind of a horizontal blinds effect running through everything, particularly fast movement, it's because you're originating from an "interlaced" source. In that case choose the source clip inside of after effects, go to interpret footage:main and select "separate fields". Most likely it will be Lower Field First, but if that doesn't look better then choose Upper Field First.

[John Ratard]

Thank you for your information.  This provides me with the experience I do not have to save time in trial and error.

The old 8mm video film cameras captured at 8fps, then Super8 at 16fps.  The Cinecap software in 3, my reply to David, must have performed the avi conversions.

[John Ratard]

I was able to get an excellent upscale with a test case, reasonable close in quality to the avi resulting from the 8mm telecine.  Unfortunately, the frame rate created abnormally fast video motion since I am confused with how frame rate is set.

In my original post I included the 12 steps I used as a guide for upscaling.  I made changes to suite my particular requirements.  Following to step 7, I could not find where to "Set the duration to double the original" nor the purpose of "then Time Stretch and enter 200" in step 9.  I imagine this has something to do with frame rate.

I used 200 in the Layer menu Time Stretch to upgrade 30fps to 60fps used in the master project I am using in which this avi video will be used. From what you have mentioned in your reply 2 regarding the automatic square pixel conversion, I probably should not have done this since the After Effects (AE) project used for the upscaling was set to the master settings of 60fps before importing the avi 30fps. The 200 time stretch must have doubled the fps again.

Please confirm the purpose of step 7 and step 9 in my original post.  Why the two steps that to me appear to serve the same purpose to change the frame rate?  Also, where would I set step 7?

[thinkshock]

I do not see the purpose of those steps to change the frame rate.

If you like the speed that your footage plays back after you transferred it, do not change the frame rate and create all compositions with the same frame rate.

If you are set on putting a frame rate change into your upscaling process, then be prepared for some math. If your source was originally 6 fps and you want to get to 60 fps then every frame should be repeated 10 times to get you to that new frame rate. At 30 fps the frame will be repeated 5 times. In any event, if your footage started at 6 fps it will only ever be 6 fps, but AE will simulate the frame rate with the new composition frame rate.

The duration setting refers to duration setting of the composition. If the composition is 10 seconds make it 20 seconds. The time stretch adjustment can be found in the layer control in your composition timeline. You may have to add the Stretch column to your timeline if it's not showing up. The instructions seem misleading. 200% will make the footage twice as fast. 50% will make the footage twice as long.

[John Ratard]

Thank you for confirming that I am using the correct method to adjust the naturally appearing motion either faster or slower using the Layer control.  The steps to deal with the frame rate I found to be valid since I discovered the following when using the originally created 8mm film at 8fps:

1. I created an After Effects (AE)  project using the settings for a Premier Pro (P.Pro) project that imported AVCHD video at 60 fps.  The purpose was to import the 8mm telecined and upscaled video to include in the P.Pro to show a before and after comparison from 1950-1960 to present day.
2. I tried 3 test renders using Time Stretch at 100, 200, 400.
3. First I tried 200 resulting in faster than normal motion.  Second I tried 100 that was even faster.  Third I tried 400 that appeared normal.

Conclusions  for my first time experience with AE upscaling:

1. Using a larger Time Stretch slowed the motion.
2. Importing the 8fps into AE was not automatically upscaled to 60fps that indicates the time stretch programming needs information to do this for 8fps.  The 400 i.e. 4 time 8 is 32 close to 29.97 that must have been upscaled from there to 59.97.
3. In my reply #3, the GSpot result that appeared to show 30fps (29.97) was not capable of interpreting 8fps in its programming nor was Cinecap capable of upscaling to 30fps in the resulting telecined avi file.

It appears that 8mm film at 8fps and Super 8mm at 16fps are not standard for present video editing programs.  Neither GSpot is capable of interpreting 8fps nor Cinecap (covered in my response #3) capable of upscaling to 30fps.

Starting at step 5 in my original post, I am still unable to find where I "set the duration to double" in step 7 and why what appears to be a duplicate method in Time Stretch.

[Rick Gerard]

John, I think you are missing some of the truths about frame rates and comps. Your workflow is really convoluted and is going to take a bunch of experimenting and calculating to get you the results you want. Let's start with some basics.

If you have footage that runs for 10 seconds and it was shot at 8 frames per second and is interpreted as 8 frames per second, and you drop that footage in a comp that is 24 frames per second the footage will still be 10 seconds long, you will just see each frame 3 times before it moves to the next frame.

If you have some footage that was shot at 100 frames per second and is interpreted at 100 frames per second and the action that was photographed was 10 seconds long and you drop that footage in a composition that is set to 30 frames per second, the footage will still take 10 seconds to play back. If you want slow motion the easiest thing to do is to change the interpretation to 30 frames per second. The shot will now run about 33 seconds and you will see every frame and time will be slowed down.

Change the fine interpretation of an imported footage that was 10 seconds long when running at 8 frames per second to 24 frames per second then the shot will now only be 3.3 seconds long. If you put it in a comp that is 24 frames per second you will see every frame but time will run 3 times faster.

Both After Effects and Premiere Pro preserve time no matter what the frame rate of the sequence or composition is. That is how all video editing apps work. If the frame rates do not match there are tools that can help blend the frames together so the motion is not so jittery, but if you want to show a comparison between 8 fps home movies (mine were all shot at 16) and 60 fps video you have to shoot some footage of the same kind of action at 60 and then put both shots in a 60 (or 30) fps timeline. A lot of home movies were run on projectors that did not have accurate speed control and many early spring wound cameras did not have accurate frame rates so sometimes the action appeared to be sped up. Sometimes when they showed silent movies, which were shot at 16fps, they were shown on projectors running at 24 fps so the action is sped up. The action was not sped up when these silent films were originally released in theaters. The speed was as accurate as the camera operators wrist was when he cranked the camera. The exception was the Keystone Cops silent films. They undercranked the cameras to give the action a funnier look.

If you have your 8mm footage telecinied (converted to video) and you want to make sure every frame is preserved you have to tell the folks doing the transfer to do so. The standard for film to video conversion in the US (all NTSC countries) is to run the film at 24 fps, add 3:2 pulldown, interlace the footage and record at 29.97 fps. If the footage was originally 16 fps they do the same thing but the pulldown is different so that time accuracy is maintained. Let me say this again: If you want every frame of film to be a frame of you need to specify that when you get the film transferred to video. You need to check your footage to make sure that there are no duplicate frames and that some kind of pulldown/ interlacing technique was not used. This is pretty easy to do by separating fields (interpretation) creating a comp that is the same frame rate as your footage, then doubling the frame rate of the comp and step through the footage one frame at a time. When I was working with a lot of film transferred to tape I had to do this for every shot.

The starting point is to get the film to playback in real time and you do that by properly interpreting the frame rate using File>Interpret Footage. All of the other workflows do not make much sense and you are making things way too difficult. Once you have the footage playing back in real time you can make accurate calculations for slow motion and fast motion effects.

Everyone that I have ever met in this business that started out shooting film understands frame rates and playback. Only about 20% of the folks that I have met in this business that have a video only background really understand frame rates and time. Don't feel like you are alone. Most do not get it and work way too hard trying to figure it out.

[John Ratard]

Thank you Rick.  I  will rethink how I have interpreted the previous information I have so far.  I will also need to carefully study the help provided by Adobe training on upscaling to understand all the variable functions provided in the software, hoping that the Adobe training also includes film prior to video.

So far I have acceptable results, to be revisited, for the test run using 8mm film from the South Pacific in high humidity.  I also need the following:

1. I am missing perhaps more advanced information on removing some black irregular dots from frames and ways to sharpen apart from Photoshop and Unsharp Mask.
2. I also need to better understand what is an automatic conversion, e.g. pixel shape or frame rates, and what is not when importing a video format into a project with a different format.

My present dilemma is how I reward more than one reply with Correct Answer to also include yours.

[Rick Gerard]

Don't worry about it. I am not looking for points, just trying to help. Video production is a very precise science and the billions of combinations of knobs you can throw in AE to manipulate pixels and time can be overwhelming. Only a very small percentage of visual effects and motion graphics artists have the depth of knowledge required to get the work done in the least amount of time with the best results. For example, it is extremely important that you understand the relationship between frame rate, display refresh rate and motion in a shot so you do not end up with a shot that is unwatchable. There is a lot to learn...

[John Ratard]

You were spot on.  I reviewed the 8mm film telecined and upscaled Export for the Time Stretch 400 setting.  Each frame was repeated 4 times.  Together with this and the upscale, the exported file size was 132GB.  This resulted from the 8mm film, captured at a claimed 8fps for a hand wound spring controlled video camera, where the telecined avi file size was only 3.42GB.  I will try again with better logic.

A helpful rating is better than nothing and very much appreciated. 

[John Ratard]

Thanks Rick for you help in understanding frame rate.  I am facing this understanding together with an After Effects (AE) program that contains an upscale function with menus that can only perform the way they are programmed.  I am not fully familiar with all this.  Also, there is sometimes more than one way of doing this where some paths are quicker than others.

The quickest way for me to learn all this is to have step by step help.  As I am finding, my last test, possibly because I made an incorrect programmed menu choice, would take 15 hours program estimate to complete for a final solution using the full avi upscale.

If possible, I need a step by step approach as shown in my original post for an 8mm telecined film at 8fps DV 720x 480 avi upscaled to 1440x1080 at 60fps HD without duplicated frames.  I see the result as 4x3 DV frames with black sides fitting into a 9x6 HD framed project.

Added at a later date: I have reconsidered my expectation to increase frame rate without duplicate frames because this would require tweening, the creation of new in between frames of changing motion from one frame to the other.