Howdy...
Late to this thread and an intermediate level editor, so might not get all the jargon correct- apologies in advance. I felt the need to share my experience with a similar problem in the past couple months.
I've just delivered an animation series and encountered something similar; because we were working remotely during the bulk of COVID and the production teams were scattered across 3 continents and 4 timezones and so that we could tweak character animation in the edit (poses, lip sync, etc), we decided to have the animation production studio supply us with the backgrounds (ProRes 422) separated from the foregrounds/characters (ProsRes 4444 + alpha).
Over and above several quality and production issues that arose (and we had to address) as a result of this workflow choice, I found that when exporting at the end to MXF OP1a with video limiter dulled a significant amount of finer details, giving the overall pic a "furry" texture and in many scenes a jagged outline around the characters, with intermittent spikes of blacks below zero.
While the "furry" pic, numerous jagged outlines and blacks spikes weren't significant enough for the series to fail QC with the broadcaster, I was really puzzled as to why the renders pre-video limiter were applied were crisp and post-video limiter were dulled/deteriorated with blacks still spiking (if only intermittently). So I ran numerous tests and worked back through our workflow to the original renders supplied by the production studio to dial-up the pic quality.
The main problem was ultimately:
- The video limiter and the ProRes 4444 alpha channel of the foreground were not friends- the jagged outlines of the characters appeared as a direct result of the video limiter.
- Whether the video limiter was applied on an adjustment layer or via the export settings, the results were the same (although the export settings seemed to have a slightly better result).
- My hunch (and not knowing the proper jargon) is that this is caused by the additional (although transparent) colour information on the alpha channel. When compressing the alpha channel with the video limiter, the pic degraded.
- Furthermore, any masking within a clips opacity Effect Control settings, generated further jagged outlines when rendered.
- The dulling of the finer details, the "furry" pic textures was also as a consequence of the video limiter compression (funnily enough, especially on the blacks within the character designs!).
As I mentioned previously, I ran a plethora of tests back to the source files:
- Different file types
- Different bit depths (with and without alpha)
- Different LUTs
- Different colour and level adjust effects
- Different export settings
- etc...
Ultimately the solution I arrived at (for our particular workflow at least) was to:
- Render out the final edit using the Animation codec including or excluding alpha at 100% quality (i.e. lossless/uncompressed*).
- Including alpha in our case, had a very slight but noticeable improvement in pic quality but, the file sizes are massive so one might want/need to work with smaller files for storage capacity/hardware capabilities so excluding the alpha might be an insignificant trade off.
- I compared these animation codec versions with other ProRes codec options and while there might've been very slight, incremental quality improvements with other codecs, the animation codec seems to have (naturally?) clamped the colours to broadcast safe without applying any level or color effects and without the video limiter.
- The other codecs in comparison, all had slight spikes of blacks and whites the whole way through every episode with varying degrees of pic quality degradation (some better, some worse) but, none of them conformed to broadcast safe colours without apply effects or the video limiter.
- Thereafter, and to be doubly sure that the final MXF OP1a render is still compliant, I'd point to andrec98422456's comment, as we did exactly this as well.
*Note: There are other uncompressed codec options that I didn't test (i.e. YUV 8/10bit 4:2:2 ) which I imagine might achieve similar results to the animation codec.
I felt the need to share our experience as there aren't many posts online specific to animation issues, I had to wade through tons of other posts on live action stuff (where there are small nuggets of info relevant to animation but not specific enough).
Hope it helps others...?
TheCar2n
AdChoices