After Effects CC 2017; Multiprocessing?

jonathanh97724603  wrote At least put the After Effects UI in its own thread.

The UI is on its own thread. That was the major change from CC 2014 to CC 2015 versions. As long as you're using AE version 13.5 or greater, you should be able to interact with the UI even when it's in the middle of trying to render something. The UI and renderer are running on different threads. What you are describing should be possible with newer versions, but if you're using CC 2014 or earlier, it won't.

And here's a little background about why we made RenderGarden.

Swordfish uses Pixar's Tractor for network rendering in After Effects, Maya, C4d, Houdini, Nuke, Handbrake, Font Syncing, etc. We went with Tractor after trying other queue's and found it to be amazingly powerful and bullet-proof, as you would expect Pixar to build for their internal productions. But it had its challenges... it was not simple to setup. We needed to hire an outside IT company to set up an NFS network, and my TD spent weeks if not months writing custom python to get everything running seamlessly. I spent over $50k in hardware/software/development to get my farm to where it is today. And I love it. But earlier this year we had a project come in which required a few of my artists to work off-site at a client's office on 6k comps while our tractor farm was slammed with other projects. Our client gave them each two Macs, so we started to think about how we could leverage the tools we made for our Tractor farm and create a mini-farm down there. Initially we created a sneaker-net solution to segment the comp to the four macs, then proceeded to think up ideas to automate this process. We didn't need a full-blown farm, just a little garden– hence the name RenderGarden. We added the Gardener tool to watch for seeds on the network, and started installing Gardener on other client machines in the office to render in the background. One of the unexpected by-products of RenderGarden was the fact that you could plant more than one seed on the same computer, which brought back the render multiple frames simultaneously feature Adobe killed in 2015. We found 2-3x speed increase on each render node, and have never seen a decrease in render speed as others here found with the old AE feature (might be how we segment chunks of your timeline and not frames). We also do not have issues with GPU, expressions, particles, etc., since we're basically using the same submit scripts we use for Tractor over the past four years. So that's the background of why we made RenderGarden.

In response to Rick Gerard's "defense" of the current state of AE: "failing to maintain a productive work environment lies squarely on the user, it's not any software companies responsibility to make sure your particular system is always working."

Well, crikey, Rick: If RMFS wasn't working for you 90% of the time, that must be your fault then. I do 3D finishing, motion graphics, and compositing. Multiprocessing in AE 2012 (yes, 2012) works 95%+ of the time for me. Another hint: anyone who produces animation (including motion graphics and VFX) knows to render image sequences. Google it if you are confused as to why this is better.

Hoorah

It works, but there is a caveat in that you need to consider your workflow.  If I'm doing a retime, the retimed footage would ultimately be reoutput to image sequences that replace the original plate. It's more efficient and avoids any potential glitches from other effects applied on a retime. Other temporal effects like motion blur, I utilize filters that don't have an issue with RMFS; from what I recall RSMB is fine for example. As mentioned, I try to avoid working with highly compressed video files as this is generally better for quality and efficiency.  RED files are output to individual frames, BMCC produces raw frames, 5D would be converted before doing any really major work, 3D renders are output to EXR, etc. Slow decompression image formats like PNG are avoided too as the decompressed frame in the buffer takes up the same amount of RAM regardless of the image file size.

Previewing (for those not familiar, RMFS preview is done with numpad 0, not space) is the most important - that's what you're doing constantly - so video output is moot there. On final export I don't recall off-hand having  RMFS issues outputting H264 mp4's, but often times once final export is needed, I'd output a HQ master - DnxHD for example or image sequences - from which other lower quality formats would be generated, often in the background. (An i7 will even compress H264's at the hardware level, though that may have an impact on RMFS; older high core xeons used to have issues here). Alot of these workarounds are also better practice in general, so more often than not, it's worthwhile and the efficiency gained from RMFS continues to demolish render times in more recent versions of AE.

Because, multiprocessing:
I did a brief test with CC 2017, versus what I'm (still) using, CC 2012. A very common scenario for me: One OpenEXR file sequence, some minimal color correction, a couple text layers. This was a comp of about 1300 frames. I did a "purge all" in both versions for fairness. Times are MM:SS.

AE CC2012:

cache full sequence:     00:40

render full sequence:    02:30

AE CC2017:

cache full sequence:     02:30

render full sequence:    04:46

Think I'll stay with the old dog. I prefer the multiprocessing.

[machine specs: Win7pro, dual 8-coreXeon/3GHz, 128 GB RAM, interal video RAID w/ typical 400MB/sec. transfer. Yes, it's a production beast]

Gutter-Fish  wrote You know, anytime someone revives this thread complaining about the speed and performance they get a slew of answer explaining how technically, earlier versions were a hack & "not real multi-threading" etc... Which may all be true.  But I think the point is that those old "hacked, half baked" versions of After Effects were faster & more stable than the current.

I can't speak for everyone, but the reason I share stuff like that about how AE is technically doing things is to try to give context for why AE changed. Yes, in many ways, for some workflows, it's worse now. But, for me, knowing the context is helpful, so I assume the same may be true for others. I would be really frustrated with AE if I didn't understand the reason why some of the stuff is being done. (Same thing for Cinema 4D and their new open beta for an update to BodyPaint. If I hadn't been told about their longer-term goals and the reason for this, less-than-stunning "update", I would have been rather upset.)

Gutter-Fish  wrote Isn't writing fixes on top of sloppy code exactly what they're doing now?

Ostensibly, no. They have been describing it as creating a new architecture. So now, it's not patches and workarounds on top of old code, it is clearing bugs out of new code. So instead of putting patches on the roof of an old building, they are solidifying the foundation for a new one. At least, that's how I understand it.

Again, it's not all that helpful to us users NOW, but for me it's nice to know the plans for the future are good.

Also, the latest couple of versions of AE have been working really well for me and I haven't opened CC 2014 in my freelance studio in months.

If you're not creating geometry with the ray-traced renderer, it's much better to set your composition to use the classic renderer.

If I were in your position and using CC 2017, I would consider saving it back a version and try rendering in CC 2014 with multiprocessing to see if it makes a difference.