Smoothing for Time Lapse

[John Ellenberger]

I am trying to create previews for walking trails by capturing a time lapse while walking the trail using a GoPro and then pulling the stills into Photoshop/Premiere do a time-lapse.  The resulting video is looks very jumpy no matter what fps I use during the Premiere export.  I used the MS "Hyperlapse" tool, back when it was free, to do the opposite--video as input and a simulated time-lapse as output.  The result from that was a lot more smooth as I recall.  This isn't surprise since they start with a *lot* more frames and I assume they are throwing some frames away and then interpolating.

My question is whether or not there is any way do smoothing across the input stills in a time lapse to try and smooth it out a little.  Have the full suite license so any of the CC tools are fair game (although I mostly know Premiere and Photoshop)

The GoPro ranges from 1 frame for 0.5 sec to 1/Minute.  I am running it at 1 sec. to produce the stills (and I walk pretty slow because I am an old guy).

[Bob_Hallam]

You will need to interpret the frames...  Here's a link that goes through the steps:

https://design.tutsplus.com/articles/2-ways-to-create-a-slow-motion-video-effect-with-photoshop--psd...

[Conrad_C · Accepted solution]

Using Interpret Footage is unlikely to solve the problem. As the linked article itself says, Interpret Footage is simply a way to properly time-remap frames shot at a high frame rate (such as 60 or 120 frames per second) to the timeline frame rate. This is consistent with what Interpret Footage does in Adobe Premiere Pro and After Effects. But the footage shot by the original poster is not high frame rate footage. It's extremely low frame rate footage with huge time gaps: One frame every few seconds or minutes. That won't benefit from Interpret Footage.

What would help here is to interpolate, not interpret, the frames. Interpolation could improve smoothness by rendering intermediate frames based on the preceding and following frames. In Premiere Pro and After Effects this is called frame blending. I don't think Photoshop has frame blending, and I also don't think it would work well with frames with so much time in between them that there are significant changes in content, edges, and camera orientation. I also don't think the Warp Stabilizer in Premiere Pro or After Effects would help much; like Microsoft Hyperlapse they need lots of similar frames to interpolate from.

Microsoft Hyperlapse might be the best that can be done here. If you look at how Microsoft does it (watch the video below — it's fascinating!), part of the reason MS Hyperlapse can can produce smoother video than the above solutions is that it not only interpolates frames, but also does interpolation of the empty areas that result when adjacent frames are not framed the same (e.g., slighly rotated) as you are walking around taking pictures along your path. To make it look like the camera was always pointed the same way instead of shaking all over the place, it also attempts to reconstruct the 3D path of the camera, through a 3D model of the world that it builds by analyzing the frames. This is all way beyond what any Adobe app I know does. Adobe video stabilization is great for what it's intended for: Normal frame rate video. But for a project like that in the original post, a hyperlapse solution is probably more appropriate. But I haven't looked into whether there are other apps that can do it.

[John Ellenberger]

Just a follow-up.  Although MS Research seems to abandoned PRO the company still gives out licenses on request.  May want to snag one before they go away too. It is a unique tool.

[JJMack]

Was you GoPro used stationary with a stabilizing mounting or on a tripod?.   Removing frames from a video  or makes a time lapse with image from a moving camera  over uneven terrain  is bound to be bumpy.  There is nothing stable object perspective will change with camera position there is nothing to align frames to. And the camera  was most likely not always pointing  the same distant point. You have a series of images  shot from different points of different areas.  A Time lapse is a series of image of a scene where the camera does not move the images capture the changes in the scene composition over time. Shoot video....

[Bob_Hallam]

Another solution is Panolapse

RAWBlend - Camera RAW Interpolation

[Conrad_C]

Unfortunately, RAWBlend will not solve the original poster's problem either. RAWBlend looks very cool…I use a similar application called LRTimelapse. Both interpolate exposure settings to smooth out exposure differences, to erase the "jumps" from changing shutter speed  or ISO during a time lapse to compensate for rapidly changing light conditions (as in a sunset time lapse). They can also remove subtle exposure flicker that affects time lapses. But these applications will not stabilize spatial jerkiness from frames taken widely spaced in time.

In short:

RAWBlend (and LRTimelapse) interpolate exposure settings only. They will not interpolate or alter image content.

Hyperlapse and video stabilization can manipulate image content through spatial warping, synthesis of edges, 3D modeling, etc.

[Bob_Hallam]

Well you probably missed the gallery of content that shows moving camera inderpolation Conrad.  It's a good solution, and a free download.

[Conrad_C]

You're right, l I missed the gallery. Now that I've checked it out, I see more of what you mean. However...

The motion isn't produced with RAWBlend, but with the other tool on that site, Panolapse. RAWBlend only interpolates metadata, but does not alter spatial information. Panolapse can, but its main purpose is to introduce motion that was not there in the first place. For example, you shot a time lapse on a locked down tripod or slider at a stationary position, and you want to create the illusion that the camera was moving or rotating by cropping the frame and animating the bigger image behind it. I've done similar "fake pan and zoom" time-lapses with Adobe After Effects, but that doesn't go as far as Panolapse which can also fake the 3D projection, making it look more convincing. And that makes Panolapse something I might want to use.

I just wanted to get that part out of the way: Panolapse synthesizes point-of-view animation where the camera was not actually moving. That is a fundamentally different problem than the original post, where there is multiple-axis motion that needs radical stabilization. I did see the example of moving over water, but I believe that was done with a locked-down camera on a boat or aircraft without the serious discontinuities of taking pictures while walking along a trail. It doesn't have the same irregularities that need to be fixed. The part at 0:42 isn't even stabilized. I looked through the support files and found no reference to motion stabilization…if you know which page it's on, I'd like to read about it.

I still can't find anything on that website that says either tool will fix orientation and framing irregularities from a walking time lapse along a path. Also, the only references I can find about interpolation are for the exposure metadata only…not for motion, orientation, or any other spatial property.

That said, I agree…John Ellenberger should download the free trials of RAWBlend and Panolapse to see if either solves the problem, even if it looks to me like neither is designed to solve that specific problem.

[Bob_Hallam]

Yes Conrad, Panolapse is was my recommendation not Raw blend.  Splitting hairs aside that solution will work for the OP's problem.