HDV --> SD DVD Workflow (DebugMode FrameServer, AviSynth)

[Dan Isaacs]

While the old thread contains a wealth of great information, I've decided to break away from it. It's too old, too long, too confusing... In summary:

Many users are aware that Premiere/AME/Encore does a "less than stellar" job of converting 1080i HDV to interlaced SD DVD. I (and others) have tried every conceivable combination of options and found none that provided "professional quality" conversions. This is largely due to the way that Premiere handles scaling of interlaced material ... Example #1 ... Example #2 ... and Premiere's failure to convert from the HDV color matrix (Rec.709) to the SD specification (Rec.601).

In an effort to solve these problems, I developed a workflow (using several third-party freeware tools) that achieves results far superior to that of Premiere and/or Encore by themselves.

I have written a guide for this workflow, including step-by-step instructions and links to download all the required tools.

This page also contains a link to a more advanced option for doing these conversions (using mostly the same tools). Please do not attempt the advanced version unless you have already implemented my "basic" workflow!

The page and linked files will be updated frequently. It is a work in progress, but should already provide excellent quality. I look forward to comments and suggestions from this community (as always).

Enjoy! -- Dan

[Dan Isaacs]

> Pr CS4 now hands *everything* off to the Adobe Media Encoder as soon as "Export..." is selected

It's really a tragedy... Debugmode is perhaps the most useful plugin ever written, IMO. I sincerely hope Adobe implements their own frameserver soon. It's really just too practical to live without. I see more CS3 in my future :(

There is one "upswing" here, though: In CS4 you can apparently:
1.) Batch process in AME
2.) Export as 4:2:2 I-frame only MPEG2 @ 100+ Mbps, so...

I am currently working on a new AviSynth batch processing tool called AVSoM. If you export all of your sequences at high-bitrate MPEG2 you can very easily batch index/script + convert them to SD AVI with my utility :)

[jefbak]

Cool! I am crossing my fingers for a cross platform version of that script. Oh wait aviSynth... darn it! I will miss my export movie option too. That is how I export HDV to quicktime animation for short projects - but I have to admit I don't know what happens to the color space exactly when I do that... Same thing goes for Motion jpegA

[Dan Isaacs]

Sorry, jefbak... PC only. Maybe it would work with BootCamp, I dunno?

How about getting a PC just so you can use all the groovy freeware that exists for it? :)

[Dan Isaacs]

I've posted an updated version of hd2sd.

I fixed something I broke in the last version (dealing with certain aspect ratio conversions) and added some new parameters (OutputWidth, ForceSDFieldRate). See the hd2sd.txt file for details.

[_arif_rasib_]

hi dan.
have finally got round capturing some native 720p 50fps clips from the ex1. files are too big to email(upto 80mb per clip). any ideas how i can send.
arif.

[Dan Isaacs]

Hi Arif -- send me an email and we can arrange FTP access or something: @ d.isaacs -- at-- comcast --dot-- net.

[_idar_lettrem_]

Dan , could you please give an example on cineform-based (aspect,1080i, PAL)workflow from HD project to SD DVD in the HD2SD context?

[Dan Isaacs]

Hi Idar ---

I got your email, too -- sorry I forgot to reply. The Cineform workflow is really pretty much the same, but I'll spell it out to avoid any confusion:

First, the requirements:

1.) Premiere Pro (all versions) + Cineform codec
2.) AviSynth 2.5.7
3.) My hd2sd function + plugins (follow the instructions in the .zip to install)
4.) Lagarith lossless codec
5.) VirtualDub (32-bit version)

The process:

1.) Export your sequence from Premiere as a Cineform AVI file using the same settings as your project (ex: 1440x1080 25 fps, upper field first). In the Cineform options, choose the highest quality, Rec. 709 colorspace (YUV), and interlaced.
2.) Create a script like such in Notepad, save it as "cineform.avs":

AviSource("c:/path_to/cineform.avi", pixel_type="YUY2")
hd2sd(OutputColorSpace="YUY2", OutputBFF=true)

3.) Open "cineform.avs" in VirtualDub, go to the "Video/Compression..." and choose the Lagarith codec --- also select "Fast Recompress" from the Video menu.
4.) Save this to an AVI file -- "lagarith.avi", for example
5.) Open "lagarith.avi" in Encore for encoding and authoring

Obviously, you'll want to use more descriptive names than "cineform.avs" and "lagarith.avi", but they should serve as clear examples.

[Dan Isaacs]

I've posted a new version of the advanced plugins, which now includes both hd2sd() and sd2hd(). [together, at last!]

There are also changes to both packages, mainly with regard to processing improvements when running the multi-threaded version of AviSynth (AviSynth MT).

You can see the hd2sd.txt and sd2hd.txt files for more information.

There were quite a few changes, so please let me know if you encounter any issues with these new versions.

[_demo_crat_]

Finally I made it! 1440x108i PAL to SD 720x576. Thanks to you Dan !
No big effort in retrospect. Simple to banal errors have hampered me up until now. But that's history.
My impressions so far after studying end results ( my eyes and none high tech vectorscopes as reference ) - - - well no big differences at the end compared to standard adobe cs3 workflow. No big differences. Sorry to say - but that's how it is. No big differences.
I believe that 16-235 restrictions is *THE MAJOR POINT*.
But Dan -I really appreciate your percistent effort - don't you ever misundertand. Good guy.
So - as you understand , I sit back here with some mixed feelings. First of all ; I believe that 16x9 is more widespread as television standard here in Norway compared to US. Second; I believe BR is more widely accepeted back here as an recognized "standard" compared to US. Thirdly; I think BR unevietably is going to "occupy" standard PP and other workflows - quality issues focused.
It migth from previous postings have been noticed that I've been sceptical to HD . But in consequence of above mentioned arguments - I purchaced and EX1 with 25p is what it's all about for now. And BR. It costs , it hurts - but as a dedicated amateur ,,, it's worth it.
So Dan - keep up your qualified and good work. Keep in mind though that evolution has always been there.

EDIT: demo crat = idar lettrem

[Dan Isaacs]

> well no big differences at the end compared to standard adobe cs3 workflow.

Can you post an example of each somewhere? I have yet to see any instance where CS3 alone does an good job of resizing 1080i, but results will be less dramatic with certain types of sources.

Also: what was the exact script you used in the end?

[_demo_crat_]

But of course Dan, I'll provide you with examples - when I know how to present appropiate *clips* proficient to justify my "verdicts".
I'll think about how to arrange this and I'll be back.

my workflow is just as suggested

"AviSource("c:/path_to/cineform.avi", pixel_type="YUY2")
hd2sd(OutputColorSpace="YUY2", OutputBFF=true) "

[Ed Cread]

Major edit here ... my remaining question is more about how to use the many features of the advanced script if I am only doing a one line script file? Seems like so many people in the thread are only using two or three items but the advanced hd2sd file has dozens.

Also - what is the general consensus: should we go to deinterlaced to end up with a progressive ouput? Or when using TFF ouput is the interlaced version generally sharper for most people?

Thanks.

Ed

[Dan Isaacs]

Hi Ed,

In general, the "critcal" parameters are OutputColorSpace, OutputFieldRate, and OutputBFF. These are the ones that govern the output format and match it the specs of a given format, codec or encoding software.

Setting Interlaced=false is also required for 24p, 25p and 30p input (when input us 50 or 60 fps, it is autmatically assumed to be progressive).

SmoothTime=false can be used when the output is a different rate than the input (such as 30p from 60i). This will cause fields to be dropped or repeated rather than blended to achieve the new rate.

For fast-motion or very sharp and detailed sources, you way want to consider setting DeintMethod=1, but this will greatly increase processing time. You can try setting maxd=8 to achieve faster rendering.

Beyond that, you may want to look at OutputWidescreen and WidescreenType, which alter the way the PAR conversions and/or cropping are performed.

It also good to use some noise reduction (via NR, GrapeSmootherAmount and FFT3dAmount) to achieve greater bandwith efficiency when encoded to MPEG-2 DVD.

Increase Sharpness when the output looks too soft, and/or increase VerticalBlur if interlaced output is flickering too much.

Other settings, like InputBlackLevel, InputWhiteLevel, and InputGamma can be used for basic luma adjustment -- particularly useful when pre-converting source footage to DV or another editable format.

There are also some debugging/diagnostic parameters like ShowHistogram, ToggleHistogramMode and PreviewPAR that can help you visualize what's going on with your material, but should be removed from your script before final output.

Keep in mind that my primary purpose for creating hd2sd was for my own personal use: I routinely downconvert HDV material to DV for editing, and many of the parameters are geared toward increasing the quality and flexibility of this process.

The Premiere "finished timeline -> DVD" workflow is really a (simpler) spin-off of the original, for which many parameters are likely redundant.

[_demo_crat_]

Big files and no ftp available, but
Dan - if you mail me at l e t t r e m (at) o n l i n e dot n o and leave your post adress I'll return

- one 40 min DVD , standard workflow ppcs3
- one 11 min HD2SD DVD, Dan Isaacs workflow

for you to evaluate/compare.

I could also add some ex1 1280x720p footage for you to play with (if you haven't already recieved this from arif).

[Dan Isaacs]

Hi Idar --

Did you not get the email I sent you (it was to your other email address)?

What I was really looking for, however, was the SAME material exported both ways. That's really the only way to make a comparison. Could you please export that 11 minute sequence as normal MPEG-2 DVD from Premiere?

> I could also add some ex1 1280x720p

Were the samples created from 1080i or 720p? Now I'm a little confused. I am also fuzzy about a few other things in your last post.

> I believe that 16-235 restrictions is *THE MAJOR POINT*.

Meaning what? The same luma range restriction also applies to Blu-Ray.

> I believe that 16x9 is more widespread as television standard here ... BR is more widely accepeted ... BR unevietably is going to "occupy" standard PP and other workflows

I'm sorry. I don't understand how any of this, even assuming that you're correct, has any bearing on this. Could you please explain what you mean?

The email I sent you has FTP login info for you. I'm all set on EX-1 footage (thanks to Arif), but thanks for the offer.

[Ed Cread]

Thanks Dan for the detailed response on the HD2SD script options. This is very helpful.

Ed

[_demo_crat_]

Dan - I'll look into my mailbox when I'm at work tomorrow. As I understand it there will be a ftp adress where I can upload some more lengthy footage.
In the meantime you can visit
http://home.online.no/~lettrem/DAN%20I/index.html for a FLV based comparison.
Original footage 1440x1080 25i, cineform

first 10 sec's (w/o sound!) as output from AME in PPCS3 - nothing extra apart from 16-235 restrictions.

Same 10 sec's thereafter undergone HD2SD - treatment

It's a demanding clip ,handheld, movement, motion blur, difficult light - but thats representative for much of my outdoor filming

Comments ?

[Dan Isaacs]

Well, it's really hard to judge much from this clip in FLV format -- and outputting to 576i (in any case) is poor intermediary for .FLV encoding. If you'd like to try something a little more relevant to this type of conversion with hd2sd, try...

hd2sd("1080i.avi", OutputWidth=640, OutputHeight=360, OutputColorSpace="YUY2", OutputFieldRate=25, DeintMethod=2, SmoothTime=true)

... and from here, you can change it to DeintMethod=1 to use better (but much slower) deinterlacing. You may also want to set SmoothTime=false to get rid of the blending.

[_demo_crat_]

upload ready for you!

[Dan Isaacs]

Idar --

The "original footage" you provided is NOT interlaced (nor is your AME output, although it is sampled as interlaced)... and why the 24 pixel black bars on the sides? Is this really an original? What kind of camera? Where's the native footage?

Something is strange about this... Please email me.

[_demo_crat_]

Dan - the "original footage" is as imported via cineform Aspect HD's HDlink 5.3( native settings for sony a1e; HQ , interlaced ) Original file 1,7 gb - and therefore trimmed to 10 sec's , exported as CFHD avi and uploaded to you with *no further modifications*
Maybe this is inappropriate ?
I'll upload another file. untouched -apart from HDlink :)

[_Kiera_Landess_]

Hi Dan and everybody!

I've been trying to implement your technique for a little while, and I'm running into a problem.

I am using Premiere Pro CS3 on Vista Ultimate.

When I try to export in Premiere using the Debug FrameServer, the loading bar appears telling me it's writing "signpost.avi", and then the status window appears, and nothing seems to happen. All the stats stay at 0% (and it says "0 client connected over the network"), and my only options are "About" and "Stop Serving".

Does this part of the process take a long time? Or is there a compatibility issue with my OS?

Any help would be greatly appreciated!

And thank you so much Dan for taking the time to explain this process, and for troubleshooting with so many users!

[Dan Isaacs]

> All the stats stay at 0% (and it says "0 client connected over the network"), and my only options are "About" and "Stop Serving".

That's the point of frameserving: While it's 'sitting there' like that, open the .avs script file in VirtualDub. Your video should magically appear, frame by frame as they're requested :)

[_Kiera_Landess_]

Oh, good grief!

I'll just run off and dye my hair blond now.

Thanks!