Maxing out RAM - Workflow tipps?

[donjojohannes]

Hi,

I have a new-ish 32 gb RAM system. I realize, I'm not up there with the pros and their server farms, but still, I'm surprised at how I max out my Ram (and get errors) when I put my CH scenes into AE. I can live with slugglishness (makes me feel young again, when I used CS3), but maxing out RAM (which in theory should not happen, because my Adobe CC should not chew up more than 80% as per "agreement")?

What I throw at my system:

I'm "guilty" of two scenes, 27 minute long, 4k res, 3 puppets in total that have a number of full body layer cycles. All this is imported as dynamic links into AE to receive no fancy effects but only 5 camera angles that switch back and forth to create a dialogue scene. Ok, I guess, that makes it 3D, but still. And I have some simple static 1080 graphics, that should add little to the load. But looking at my Task manager, the problem is not with AE anyway. AE has no reason to complain. It even gets it own SDD to cache. But it seems that CH running in the background is hogging the memory to provide the dynamically linked data.

What can I do, to get the system out of the red zone and realm of pop up warnings (+freezes)?

Workflow suggestions anyone?

note: If that was my the only episode with these characters, I'd just split scenes and make it more digestable. But I have 15 episodes with more than 7,5 hours of dialogue in total to work through and keeping things tidy would be nice. Any tipps welcome.

Cheers,

Johannes

[donjojohannes]

Troubleshooting (and failing so far)

I tried to strip the puppets down to basically just head movements without any cycle layers. This brought the PSD files down from 145 mb to 2 mb. I just did visimemes. No other recordings. Then I imported into a new comp in AE (purged all memory).

How did it affect dynamic linking? Not at all. Character animator still hogs memory in the dynamic linking process.

[oksamurai]

I can ask the rest of the team for more specific advice, but to me it sounds like this is pushing what we're currently capable of. 4K at 27 minutes and 3 puppets is a lot of data - my usual recommendations for performance are merging non-tagged / extraneous layers, turning off head turns and dangle until you need them, lowering framerate, and lowering puppet dimensions/resolution. The "turning behaviors off" trick works a lot for me, where I'll toggle the eyeballs off for certain intensive behaviors until right before a final export, giving me a more bare-bones version of my character to comp in AE. But it sounds like for you that even turning everything down to the basics is still running into issues...

[donjojohannes]

Hi Dave,

Yes, unfortunately making the puppet simpler does not change the basic problem. It almost feels like it is the length of the scene that causes the problem. As if the RAM required by the linked file depended on where in the time line you are working. The more you come to the end of the sequence (even if you start there) the quicker RAM filles up. But that might be a faulty observation.

My adapted workflow looks like this at the moment. I'm playing around with (2) at the moment. I have yet to attempted point (4), but assume it will work if I don't scrub through the timeline filling the RAM before export:

1) Do all within Character Animator.

(While CH takes a while to prepare the scene, it does manage. The Problem comes in with dynamic linking because with 2 scenes it runs two instances of CH and that quickly maxes things out).

2) Export the 2 scenes as videos. Here I'm currently testing, which is fastest export without creating huge 70 gig files. I'm not looking for quality really, just a fast, reasonably sized export to be used as stand-ins for the AE Comp. Any recommendations on format and codec for animation? Transparency would be useful, but if not, I can do a rough key

3) Setup all cameras and mographics in AE using the exported videos to tune my cuts

4) replace the videos with the dynamic links copying properties from the former to the latter

5) export via AME

Downsides: Encoding the videos from CH is a painful step in between and wait. To create one scene in MP4 low, low quality yields a small file but even the simpler scene (the one, with a single puppet) took 3h 20min. I expect double for the complicated scene. Again, if you have suggestions for a good time/file size ratio format and codec, it is much appreciated

Upsides: It works. AE might still eat my RAM but it will not crash the PC, because it will stay below the max assigned to in the preferences. Further:  in my case the RAM previews are real time basically which should help setting up the puppet environment. It won't offset the rendering time of step 2, but...

BUG REPORT?

Not having too much prior experience with dynamic linking: In the preference menu of Adobe products I can set a max value of how much RAM they are allowed. Obviously when I run out of RAM something is wrong here. I can think of two things:

1) CH is not (yet) part of the suite. Therefore it is not part of the array for which I set the max RAM

2) Or it is because in general software instances called up through dynamic linking are not considered.

Whichever is the case. A resolution would prevent maxing out RAM to the point of crashing (no BSOD, but the display driver crashes among other things and I need to do a hardware reset)

[CoSA_DaveS]

Sorry it's not working well for you. Indeed Ch does not yet take part of the memory-usage pref in AE/PPro etc.

Because of way the animation is generated, in order to render a frame for dynamic link, all previous frames must be processed (but not rendered) first. These results are cached so it doesn't get insanely slow when retrieving frames out-of-order.

It may help to split the scenes in Ch into shorter scenes, though as you've noticed each scene generates a new background process, so this is mainly helpful if you're not working with the entire 27 minute length at one time in AE.

If you're willing to send us the project we can take a look and advise further, and it could help us find bugs in Ch.

[donjojohannes]

Thanks for your your clarifications and tips.

Today I shelled out some money to upgrade my system from 32 to 64 gig of RAM. Being in the mountains in Italy, shipping will take a week (plus a 5 mile walk down to the valley to pick it up 🙂 ). When I get it I can report back on how it affects my workflow and I hope this will get me through the project. Being able to make minor adjustments through dynamic linking with only one instead of two long exports shoult save time that will offset the investment.

If I still run into problems, I can send you the project. Thing is - being in the aforesaid mountains - internet is on a slow metered connection and sending large files is a last resort and over-night procedure.

I'll let you know how I fare,

Johannes