Known Participant

4k Footage in 1080 Sequence Set to "Scale to Frame Size" will Black Out Other Clips on Relaunch

Forum|Forum|2 years ago
June 9, 2023
8 replies
2623 views

Hello Adobe Devs and Community,

On a recent project my team and I discovered a bug that was incredibly disruptive to our workflow. Brief background - the project was a conference style presentation with heavy use of picture-in-picture design. We encountered this issue in our workflow when executing some of these picture-in-picture looks with 4K footage.

Issue - Our sequences are set to 1920x1080 at 29.97p (framerate doesn't seem to impact this bug as far as my tests can tell). We were placing footage of the presenters on top of a motion background. Most of our footage was 1920x1080 to match our sequence size, for these we simply scaled the footage down to the picture-in-picture design spec and everything worked fine. The problem arose with footage that was shot on-location using Sony FX3's at 3840x2160 29.97p. To work with the 4K footage, when bringing the clips into our sequences we would utilize the "scale to frame size" toggle to conform them to 1080. This allowed us to use the same scale values as our 1080 footage and made the process easier. Unfortunately, this is where the bug showed up. After setting the 4K footage to "scale to frame size" and scaling it down over the motion background, when closing and re-opening Premiere the motion background and any other asset on a track below the 4K clip would be blacked out. The blacked out areas remain blacked out both in the program panel and in exports until fixed (see fixes/workarounds below).

Adobe Premiere Pro Version: 23.4.0 (Build 56)

Operating System: Primarily Windows (Windows 10 Pro 22H2), but I was able to recreate the bug on MacOS (Ventura 13.4) as well

System Info (Windows):

CPU - AMD Ryzen 9 3950X
GPU - Nvidia RTX 3080 Ti (Studio Driver Version 528.49)
RAM - 64GB @ 3200 MHz
Boot Drive - Samsung 970 EVO Plus 1TB (Connected via motherboard m.2 slot)
Media Drive - Samsung 860 EVO 2TB (Connected via motherboard SATA)
Media Drive 2 - QNAP NAS attached via 10G network cable

Video Format: The background file is a ProRes422 in a .mov rendered from After Effects, 1920x1080 29.97p. The footage that was scaled to frame size then used as the picture-in-picture seems to cause the bug regardless of video format, the original footage that we noticed the bug with was shot on a Sony FX3 at XAVC-I 4:2:2 10bit in a .mp4 at 3840x2160 29.97p. I was able to recreate it using both an XAVC-I 4:2:2 10bit .MXF clip from a Sony FX6 (3840x2160 @ 59.94p) as well as a ProRes422 3840x2160 59.94p .mov from an Atomos Shogun 7.

Workflow Details: Most are laid out above, the bug appears in a standard 1920x1080 sequence created with the new item button. We were able to recreate the bug in both a team project and in a standard project.

Steps to Reproduce:

Import a 1920x1080 "background" clip and a 3840x2160 "footage" clip.
Create a 1920x1080 sequence
Drop your 1080 "background" clip onto video track 1
Drop your 4K "footage" clip onto video track 2
Toggle the "Scale to Frame Size" setting on the 4K "footage" clip (right click > scale to frame size)
Scale down the 4K "footage" in the effect controls panel so it is now a picture-in-picture (I used 78%)
Reposition the 4K "footage" in the effect controls panel (I used [960,524])
Save the project
Close the project and/or quit Premiere
Re-open the project

(video of bug recreation in case embed doesn't work: https://vimeo.com/834778457)

Expected Result: One would expect to be able to see the 1080 "background" clip behind the 4K footage clip as it was when the project was last open.

Actual Result: Entire canvas except for the 4K "footage" clip is blacked out.

Workarounds/Fixes: While dealing with this bug we found a couple workarounds. You're able to un-blackout the background by affecting certain attributes of the 4K "footage" clip, scale and position adjustments will fix the problem. Un-enabeling and re-enabeling the clip also works, HOWEVER, this cannot be done as a batch, you need to manually unenable and re-enable every clip individually. These workarounds are tedious though as you need to re-do them EVERY TIME you close and reopen the project. We found that a more persistent solution that eliminates the problem when closing and reopening the project can be implemented by dropping a crop effect (but not changing any values) on each 4K "footage" clip that causes the issue. For some reason this makes Premiere happy and the bug no longer occurs.

(video of bug workarounds in case embed doesn't work: https://vimeo.com/834778479)

Weird Quirks: While re-creating this bug to gather information for this write-up, I found that it seems to occur in possibly very specific circumstances. It doesn't always occur at every possible iteration of scale and position of the 4K "footage" clip. It's almost as though the 4K "footage" clip needs to be on top of a specific portion of the canvas for the bug to occur. Our original design of having the 4K "footage" clip at 78% scale and [960,524] position will cause the bug, but if you were to use values like 30% scale and [339,248] position, the bug doesn't occur.

Please let me know if any additional information is needed or would be useful, thanks!

-Andy

R Neil Haugen

Legend

Hey, great response. The details all actually matter to fully understand the reasons for your workflow patterns.

And yea, it would be nice to get a predictable behavior.

Everyone's mileage always varies ...

Andy_951Author

Known Participant

Hey @R Neil Haugen!

Right, when we first encountered this bug, we were intentionally using "scale to frame size" to bake the scale into the footage to make it match the sequence frame size because our Picture-in-Picture layout templates had scale values based on 1920x1080 footage (the size of our sequence) rather than 3840x2160 (the size of our actual footage). In the initial iteration of this project we never needed to scale the footage to be any larger than 1920x1080 (no digital punch-ins), so conforming it to 1920x1080 using "Scale to Frame" allowed us to use the originally designed picture-in-picture values on our scale parameter in the ECP, since it remained at 100% after using ScaleTo. Using "Set to Frame" would make our scale parameter 50%, which would lead to some math in order to match the picture-in-picture template, albeit it's VERY simple math (just dividing the template scale by 2 to get the value for 3840x2160 footage), but when working with a team of editors trying to follow the same parameters, adding any hiccups to a workflow can be a real headache on the coordinater's end, hence the persistence to use Scale to Frame.

In the recent iterations of this project, we do just do the math of dividing the template scale by 2 and using that value because lately we have needed to do digital punch ins (which we achieve by using a transform effect + crop effect so we don't need to touch the original scale parameter and the picture-in-picture remains in its designed frame, pretty handy! Makes Premiere REALLY upset if you scale up the transform effect too much further than the original footage's 100% mark though.)

I mostly keep this bug report alive less because it's something we actively use and encounter, but moreso because it's something that's STRANGE and definitely a bug. I agree that 9 times out of 10, Set to Frame is preferred since it doesn't actually bake the scale into the clip, but there are occasionally projects that have workflow benefits from using Scale to Frame - for example, we do a lot of work with powerpoint slides that occasionally get rendered at wacky resolutions (but still 16:9), easier to conform those to the sequence size using Scale to Frame because we'll never need to scale them again after the fact except for needing to scale them DOWN for a picture-in-picture layout where starting from a scale of 100% can be easier to wrap one's head around.

Apologies for the long-winded answer, hopefully it makes sense!

To answer your final question, the bug doesn't seem to show itself if all scaling is done using Set to Frame / the Scale property in the ECP. I think it's isolated specifically to the Scale to Frame toggle.

R Neil Haugen

Legend

I'm curious why you use Scale to Framesize, rather than Set To?

Scale To rasterzized the image on that sequence to the sequence framesize. ALL other scaling is done from that rasterized image, not the original pixels.

Set To Framesize essentially invokes the scaling options on the Sequence ECP, and any further scaling happens from the original pixels.

Like most users, I avoid Scale to like the plague.

So I wonder if the issue would occur using Scale To rather than Set To?

Everyone's mileage always varies ...

Andy_951Author

Known Participant

UPDATE: Almost a year since the initial bug report and I'm back once again to share that this issue persists in Premiere v24.3.0, workarounds still work to avoid the issue fortuantely!

Andy_951Author

Known Participant

UPDATE: Another Premire update, another round of keeping this bug report alive. This issue persists in Premiere v24.0.0, workarounds still seem to work just fine though!

Andy_951Author

Known Participant

UPDATE: To keep this bug report alive and up-to-date, this "background blackout" bug is still present in Premiere v23.6.0.

Fortunately, the workaround of putting a crop effect with no values onto the clip DOES still solve the issue, so we're proceeding with that for now.

Andy_951Author

Known Participant

Hey @Kevin-Monahan,

Thanks for replying my bug report, it's nice to see that Adobe has people monitoring these forums frequently!

I just went through your suggestions to try and prod the issue some more to see if anything resolved the issue and I figured I'd report my findings to help provide the devs and community with more data as well as clarity on what's happening:

Suggestion - Update NVIDIA Graphics Driver

Unfortunately the bug persists through the graphics driver update, though this isn't terribly surprising since I'm also able to recreate the bug on MacOS (Tested on an M1 Max Macbook Pro 16" w/ 64 GB of memory).

Suggestion - "Use Set to Frame Size" instead of "Scale to Frame Size"

This DOES solve the issue, which seems to make sense since as I understand it, "Set to Frame Size" simply adjusts the clip's scale value in order to fill the frame. It's not an on/off toggle like "Scale to Frame Size" is. Our workflow utilized "Scale to Frame Size" because of the convenience it brought by not altering the scale value at all which let us use the same presets for all of our footage despite its starting resolution. If this bug persists, we could switch over to simply creating new presets that our editors could choose between based on the resolution of the starting footage. Unfortunately this method is slightly less "dummy-proof" as it requires the editors to be constantly aware of the resolutions of their clips and alter which preset they use based on that as opposed to following the same procedure every time. Not the worst outcome, but I always try to remove potential human error from these sorts of things where I can (helps me avoid getting the same issue/question repeatedly from the editing team haha).

Suggestion - Transcode the Sony XAVC-I footage to ProRes

I went one step further and re-encoded the Sony footage to both ProRes422 as well as a 20Mbps CBR H.264 via Media Encoder. Unfortuantely both re-encoded files caused the bug. Doesn't seem to be the codec/container of the clip as far as I tested.

Thanks again for your response and suggestions Kevin! There's definitely enough workarounds for this bug listed in this thread to keep everything on our end on-track and hopefully help anyone else who encounters this problem!

Have a good one!

-Andy

Kevin J. Monahan Jr.

Community Manager

Hi @Andy_951,

I read your bug report. Great job on the bug report. That is appreciated more than you can know! I am in support and not a dev, but I'll take a crack at it.

I would first see if it might be related to your GPU. See if updating your video driver to the one here: https://www.nvidia.com/en-us/drivers/results/205036/

One other possibility is the "Scale to Frame Size" function. I really think it isn't a good one to use, however, "Set to Frame Size" always works better when it comes to scaling an image. If you need to abide by certain settings, you can make a preset for the new settings. Try an experiment there.

Long shot, but it could also be related to the original footage. You might try transcoding the Sony footage to ProRes as a test.

It's possible, also, that there's a bug in Scale to Sequence or some other part of Premiere Pro.

With that said, I hope the product team can reproduce your error and fix it for you. While a response is not guaranteed, you might try the workarounds I suggested first to see if that gets you to the finish line more easily.

Have a great weekend.

Thanks,
Kevin

Kevin Monahan - Sr. Community and Engagement Strategist – Adobe Pro Video and Audio

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded