Auto Reframe Sequence making strange decisions

[joeleo321]

Hi there! 

I'm using auto-reframe sequence to change a 16x9 sequence to a 9x16 sequence for social media purposes. I've used auto-reframe sequence before in the same way and have had good results with it but for the past 5-10 or so times I've tried it I've got these strange results where it doesn't track with the subject's face. 

I'm attaching a clip so you can see its behavior. It's a pretty straightforward clip, a face is clearly in view and I've done this in the exact same setup before but I can't figure out what I'm missing. I know I can overwrite generated keyframes, which I've done before for the occasional hiccup, but it's like it sees a ghost in the clip and is off the whole time. Obivously, if I have to overwrite all of the keyframes it's not worth using it.

To add more details, I'm right-clicking on the sequence in the Project panel and selecting Auto Reframe Sequence, not applying the effect to individual clips. 

I'm really hoping it's just a bug that will get sorted soon or I'm doing something wrong, that would be even better, because I really like this feature for quickly pulling selects and reframing for social media.

This is my first time on these forums so be gentle with me. Any help is appreciated!

[edwin.grospe]

I've updated to version 23.1.0 of Premiere and I noticed that Auto-reframe does not work like it used to. On previous versions, it works most of the time, now it just scales up the clip/s and does not do any automatic keyframing  It did some automatic keyframing, but the adjustments are no more than 1pixel. There was also a time that it did work but the subject is not centered so I have to offset the reframing.

Windows 11 22H2

Premiere Pro v 23.1.0

[Stan Jones]

joeleo,

I can see how Auto Reframe could be a massive time-saver for social media work. And how this odd behavior would make it useless.

I have never used it. So I gave it a try, using a clip from an Adobe Beta tutorial related to the work on "assemble rough cuts from transcript":

https://community.adobe.com/t5/premiere-pro-beta-discussions/text-panel-enhancements-part-2-assemble...

The Kerstin interview is the perfect material: the interviewee is mid-screen, head and shoulder shot, very little movement, none of which moves the head very much at all. Auto Reframe settings at default.

The general result is excellent: Frame is maxed vertically, and I could not crop horizontally any better doing it manually. EXCEPT that 9 times in the 3 minute 46 second clip, the frame moves completely off the subject (onto a wall with 4 pictures) and then back again. Each move is only a few seconds. I don't see any setting that would help a problem like this, but I did not experiment. I see there is a keyframed option for offset, which I assume can make adjustments. But in this material, I'd be better off just making the whole adjustment manually.

So in my one sample, I get results just like yours.

No need to be gentle with you! You provided very good information, and a good illustration of the problem. Always good to also provide operating system and exact version of PR.

I'm Win10. I thought I tested on PR 23.1.0, but I didn't: I am on Beta 23.2.0.48 - that's why I had that clip handy.

A moderator may change this to a bug report: that's what I think is warranted. In the new bug/feature system, you just make a regular forum post, and select the option for bug.

Is this a known issue? Can anyone provide a workaround? @Kevin-Monahan @Trent Happel 

Stan

[Kevin-Monahan]

Moved to Bug Reports. Thanks, @Stan Jones, for verifying.

Kevin

[michelle27336938wqky]

Hi have the same problem, now the editing time became double due to this bug on the auto reframe. @Kevin-Monahan when will you adobe fix it?

[Kevin-Monahan]

Hi have the same problem, now the editing time became double due to this bug on the auto reframe. When will you adobe fix it? @Kevin-Monahan 

I am sorry for the frustration, michelle. How frustrating. I cannot say when the issue will get fixed, as I am not on the Premiere Pro Team. Hopefully, we can get a response. Thanks for upvoting the issue. I will continue advocating for the fix on behalf of the community.

Kevin

[Lance Pellerin]

Same problem, at times it doesn't even center the subject.

[Stan Jones]

Quick test. Not fixed as of Beta 23.4.0.15.

Stan

[ryancgreen]

Same thing happening to me. It only writes the first couple keyframes and shifts the video over, and that is it. Used to work great. Windows 10

[ryancgreen]

Any word on when this will be fixed?

[Kevin-Monahan]

Hi Stan,

Thanks for the feedback. I tried to reproduce this problem with the Kerstin interview but I could not on v.23.3. I am on an M1 MacBook Pro, though. Perhaps it is OS related, but I'm not sure. I'll try to test it out on my PC to see if it is a PC only bug. Are others on this thread on macOS or Windows?

Thanks,
Kevin

[Kevin-Monahan]

Hi Ryan,

Are you on macOS or Windows? Is it possible to share a sample clip with me? You can PM me the link, if you would like to keep it private. The problem is that I cannot reproduce the issue on my end.

Thanks,
Kevin

[Spencer5E9C]

I am having the same problem. Auto reframe is no longer working even with very simple clips with a locked off camera and a subject that isn't moving much. It completely misses the action. However, it occasionally DOES work. There is something that can throw it off. I will provide a demo clip. One one occasion it worked, and kept the subject in frame. However, on subsequent attempts it failed and I can't reproduce the working example again. I am on Windows. I have tried with a ProRes 422. mov as well as a .mp4. I am reframing from 16:9 to 9:16. Premiere version 23.2.0 (Build 69)

Here is a demo clip with a project file showing the failed auto reframe. This link will expire in a week. https://we.tl/t-0z74fymzG2

[Spencer5E9C]

I wanted to add that I reinstalled an older version of Premiere (22.6.4 Build 2) and it works in that version.

[Lance Pellerin]

From my testing, it seems to be an Nvidia problem. Changing Premiere Pro versions doesn't do anything. 

I had an RTX 3080; it would do it. If I disabled my 3080 and used my CPU's GPU (Intel), it would work just fine.

I bought myself a RTX 4090, still having the same issue.

[ryancgreen]

I was wondering that too. I recently updated from a 1070ti to a 3060ti, and come to think of it, I think that is when I started seeing the issue.

[ryancgreen]

WELL: That was the problem. I just happened to check for a latest driver update from Nividia, and there was one for the 30x series. Installed that and Auto Reframe is now working correctly again.

[Stefan27979436n5kw]

I have the same issue, just that it looks even worse. I thought I was doing something wrong.

[Stefan27979436n5kw]

Here is how mine looks (by Auto reframe sequence)

Same even after getting the latest driver from Nvidia

Hope this will be sorted soon

Stefan

[Francis-Crossman]

Hi, I'm the Product Manager for Premiere Pro.  I just heard about this issue.  It's entirely possible that GPU drivers could cause the issues you are seeing, and I'm glad to hear that updating the drivers was a solution for many of you.  Auto Reframe uses AI to identify objects of interest and then try to keep them in frame.  It uses the GPU to compute the AI model.  So if there are issues with the GPU or driver, it makes sense to me that weird things would happen.  We didn't make any changes in this area recently, so I'd be very surprised if we introduced a bug.  I downloaded the clip that @Spencer5E9C posted, and I got good results (but I'm on Mac).  One tip I have for content like that is to use Scene Edit Detection first to create cut points.  That way, Auto Reframe doesn't have to calculate movement across scene, which can really throw it off.  The other tip is to try using the "Slower Motion" preset for this type of content since it's mostly static.

[Stan Jones]

ryancgreen,

What driver version are you using?

Stan

[ryancgreen]

The newest one releases on March 22-23. 531. Sorry I don’t have the exact
version as I am not at my Pc at the moment I am however using the Game
Ready not the Studio version. --
-----------------
*Ryan*

[Stan Jones]

This fixed it for me, updating the driver from 527.56 to 531.41.

@Francis-Crossman Thanks for the explanation. I can't get my head around it, given that it was apparently works in an earlier version with a driver that doesn't work in the newer version. lol. Some things are still just a Black Box.

Stan

[joeleo321]

Hey all, I'm checking back in on this after reporting it back in January and can also confirm that updating my GeForce Studio Drivers to 531.41 has solved this curious case. Auto reframe sequence is working like a charm and I couldn't be more chuffed to have it back in action in my projects. 

I have a RTX 3070ti, running Windows 10, and 23.2 for Premiere.

I will probably be making a YouTube video about this to help out folks who don't use the forums. This was my first time joining in on a forum discussion and really enjoyed getting this solved.

Thanks!
Joel

[erikw94271635]

I use the auto reframe all the time on my MacBook Pro 2019, but it just won't work on my new mac mini m2. A lot of work tot transfer all my videos back and forth.

[David_CH_76]

Hi

We have the same issue on our Win10 PC's on Premiere 23.2 and i installed the latest Driver for our Nvidia RTX2080. But reframeing does still nothing. Not even an Sequenz in 9x16. Really nothing. It's a Pain to convert all the 16x9 Videos by Hand to 9x16. Could you give me a tip how to solve this? We have more then 10 of the same edits and its horrible. Thank you. Dave