Known Participant

Question

Render Farm Possible in Premiere?

Forum|Forum|14 years ago
July 26, 2011
5 replies
43787 views

I multi-machine render in in AE all the time. (For those who don't know, that's using several different computers to render the same project faster.)

Can you do this in Premiere? The closest solution I've come up with is rendering out and assiging frames to TIFF sequences on 3 computers like this:

Computer 1: Frames 1-2000

Computer 2: Frames 2001-4000

Computer 3: Frames 4001-6000

But if computer 1 finishes first, it doesn't go on and help out its buddies, it just sits there like the last intern I had waiting to get fired and not looking around for anything to do, like move the 30 c-stands and sandbags to the grip truck instead of drinking pop and watching all of us work...

Oh yeah, render farming.

In AE you can click it to "skip existing files" (or somethign to that effect) so that if AE sees that frame 1001 is done already, it automatically skips to frame 1002, and on and on until every last frame is processed.

Anything like this in Premiere that I've HOPEFULLY just been overlooking ?

This topic has been closed for replies.

A

Alexander Skobtchenko

Participant

The best option I've been able to find, is to export your Premiere Pro Project via Media Encoder (which can also be done for After Effects), and setting up the watch folder there.

Basically using Media Encoder for Network Rendering:
Using Adobe Media Encoder as a Remote Render Farm for Adobe After Effects (schoolofmotion.com)

I've sought advice from Adobe re: Distributing Exports across Multiple Render Nodes, so as to render more quickly by utilising many more machines' resources, from the network.
And they've said that this solution also works with Media Encoder, not just for After Effects:
After Effects User Guide (adobe.com)

I'll be playing around with this over the next little while. Hopefully it'll allow us to accomplish all of our Render Farm needs!

H

HarleyTDavis

Inspiring

Here's a solution for you,

dynamically link your sequence into a new comp in AE, then render farm that, and compress\wrap it on the back end.

CharlesW

Participant

"dynamically link your sequence into a new comp in AE, then render farm that, and compress\wrap it on the back end."

Hi everyone, first I must say that I would love to see a solution to this as we are moving into 4K productions.

HarleyTDavis I personally haven't had good experiences linking to AE and rendering in-line. Granted the project was full HD, 30 minutes long and involved about 3 minutes of 12K-sized video with animations in 3D space, lighting effects and such. AE rendered fine outside premiere but the dynamic link caused failure every time (yes I tried it twice... I like the mystery of it and the let-down after) on a robust Dell Precision T7810.

--Possibly a solution if its a small ratio and/or short length... As for me, I'll render and render and compile the edit on premiere until the Lord comes with His new creation... a place where render farms are plentiful and fruitful and compiling takes only a word.

H

HarleyTDavis

Inspiring

‌did you store the premiere project, sources etc on network drives that were mounted on all machines? How about drag n drop of sequences onto AE? that last one was a big one for me when rendering anything from premiere in AE in CS6 and that was with a single machine. all machines might also need premiere installed, for the engine to be used, but ive never not done that. it has worked across macs with Ae cs6 and one set for render only. make sure typefaces and plugins match across both ppro and ae, check licensing. ive only had it work perfect 1time with all above suggestions, and probably one or two ive missed.

H

HarleyTDavis

Inspiring

Here's the straight poop,

I don't know if they'll ever get render farming up and running for anything outside of 3d effects, but the reason it hasn't been built yet is the same reason render farming took so long to develop in the first place: the workflow.

Render farming doesn't output a single file. Why?

It uses the FRAMES and outputs a number of them from each machine in the farm, all to the same location (on any chosen machine). However, these FRAMES are simply picture files. That's right guys, just single picture files. The same is true for video files really, that they are made up of nothing but picture files being rendered to the screen at super fast speeds, but they are all bundled together. In compressed video, you lose some of the quality of the images in favor of saving speed and space, but with lossless codecs you are actually processing every frame (or every single picture). This means that you'd have to add that processing step to the workflow for render farming.

Steps originally:

Comp A process these files as images from frame x to frame y and store in folder z

Comp B process these files as images from frame v to frame w and store in folder z

and so on for every machine

Added steps for video rendering:

comp A frames x to y scan and compress to file a

comp B frames v to w scan and compress to file b

and so on through each machine until all are finished

finally: selected comp combine files a to (whatever)

However, with this function you may get extra I-frames or B-frames, and you'll end up with jitter in the final product. For this reason, the logic holds that instead of processing the video compression on multiple machines and combining files at the end, full video render farming should use a DATA PROCESSING BALANCE algorithm, allowing one machine to pass data between the processors of other machines while handling the file build on it's own. Separating the file on the fly (as it is being processed) is a more useful tactic. Start with I-frame A and I-Frame B and send the processing to Machine a, take I-Frame B and I-Frame C and send to machine B and so on and so forth. This allows there to be I-frame overlap, and all the writing is still done with a single machine, while the processing is done across several.

Of course, if you use the first set and change it so that the i, b and m frames over lap, then match them when putting the files together into one, you could, in theory, process it much the same as standard render farming. The second method is only faster due to the fact that all the processing would happen on the fly, and the file would be all one file right away, removing the step of putting several files together, and it wouldn't matter which sections were finished first, how many machines there were, or how many frames overlapped, as the overlapping frames would simply overlap and all the others would fall into place. The only bottleneck would be drive speed and fragmentation of the file, which could be a last step in finalization or an intermediate step along the way. If you use a dedicated and allocated space for the storage, on the fly is the way to go; it also works if you simply run a defrag pass over the file when a section overlaps, thereby moving other sections forward or backward in storage so they line up properly; however, this could just as easily work at the end of the whole process.

From a logical standpoint, it works. From a programming standpoint, it's a lot of work, and it's hard on a system. Personally, farming out a few effects and then wrapping them in a lossless file for use while editing is easy enough. And the quality of the preview is amazing. Rendering out to a single full video file is saved for last or for the end of the day when go home to my family; that way I don't have to bitch about how slow everything was. I can just relax and let the PMS'ing wife yell about how nothing is going right, let the kids yell "he did this" and "she did that", and enjoy the fact that my computers are more obedient. They do what I say without slamming doors and pouting.

R Neil Haugen

Legend

Thanks ... that's quite a good explanation so that I can understand it. Love learning things which is part of why I hang around here with all the work & missus & family and such ...

Neil

Everyone's mileage always varies ...

H

HarleyTDavis

Inspiring

Yep. Video files are just picture files that change one frame to the next very quickly... ...exactly like the old celluloid film. Compression is a different animal. Rendering and compression are often interchanged as far as terminology goes, but they can have different meanings. Rendering gives you a product to use, whether it's one file or several, it only processes the frames and spits them back out. Compression is a second step. Remember the old JPEG standard? Well, think of it like this... ...when compressing a JPEG, the photo is analyzed for color changes, and then shrunk down based on the analysis, and on the resize factor. With a movie, you have several JPEGS (hundreds to thousands for a few minutes of video). When you shrink the aspect ratio or resolution (pixel size), you do the same as JPEG compression would. When you compress the video data, whether you resize or not, you are using other algorithms to catch the changes from one frame to the next over a section of them, and then storing the first image and keeping only the changes from the other frames. This results in less data actually being stored, and in some players, less screen motion, which makes playback (occasionally) smoother. However, it also can result in Jitter where motion in the video is full of the full frame images and it can degrade those areas excessively. When you render farm, you don't want to compress your output. You just want the images... Here's a solution for you... If you want a solution for premiere... ...Send your sequence to after effects, and render farm it from the ae render engine, and set the quality of each frame (I warn you, this will take a long time, as it is not compressing anything). When you are finished, you can compress that to whatever format you want, and it should be relatively quick. Going from one compression format to another is slow, due mainly to the fact that the file may need to be uncompressed and recompressed on the fly. But uncompressing completely with a render out of AE, then compressing to your final should hold quality and be faster in the compression phase, and it will allow you to speed along the render phase using multiple machines.

Say hi to the missus and the family

E

excited_Genie16B8

Legend

Can you do this in Premiere?

Not yet.

https://www.adobe.com/cfusion/mmform/index.cfm?name=wishform

able123

Inspiring

=============

But if computer 1 finishes first, it doesn't go on and help out its buddies, it just sits there like the last intern I had waiting to get fired and not looking around for anything to do, like move the 30 c-stands and sandbags to the grip truck instead of drinking pop and watching all of us work...

============

Howdy, fellow grip ! hearing this is like a ray of sunshine here for me.... despite the problem mentioned....as I have not met any grips here yet over the past couple years of making everyone angry at me for having a dumb sense of humor and a brain that is close to a 'backward clam' ( like a clam that was dropped on its head as a baby ).

One rule is to not get off subject too much...so I am already shaking in my rockports.... where you working ? on what ?

ps... if you have walkies you can sometimes call the members of crew in a daze and say stuff like , " Hey YOU ! Move the equipment to the TRUCK NOW "... sometimes works OK... take it as it comes so to speak...but be polite and patient.... film people are weird to begin with so you never know what someone might be smoking etc.