HDV --> SD DVD Workflow (DebugMode FrameServer, AviSynth)

[Dan Isaacs]

While the old thread contains a wealth of great information, I've decided to break away from it. It's too old, too long, too confusing... In summary:

Many users are aware that Premiere/AME/Encore does a "less than stellar" job of converting 1080i HDV to interlaced SD DVD. I (and others) have tried every conceivable combination of options and found none that provided "professional quality" conversions. This is largely due to the way that Premiere handles scaling of interlaced material ... Example #1 ... Example #2 ... and Premiere's failure to convert from the HDV color matrix (Rec.709) to the SD specification (Rec.601).

In an effort to solve these problems, I developed a workflow (using several third-party freeware tools) that achieves results far superior to that of Premiere and/or Encore by themselves.

I have written a guide for this workflow, including step-by-step instructions and links to download all the required tools.

This page also contains a link to a more advanced option for doing these conversions (using mostly the same tools). Please do not attempt the advanced version unless you have already implemented my "basic" workflow!

The page and linked files will be updated frequently. It is a work in progress, but should already provide excellent quality. I look forward to comments and suggestions from this community (as always).

Enjoy! -- Dan

[Dan Isaacs]

> Should I NOT be doing upper fields first??

Well, in general you just need to tell your MPEG2 encoder that you are feeding it TFF and to flag it as such.

What are you using to encode? If you are using CCE, I believe TFF is the default, but check to make sure it is set to this. Make sure it is not shifting the video up by one line (it has an option to do this to reverse the field order of the source).

If you are using Encore to encode, I'd recommend BFF instead. Encore may assume 720x480 video is always BFF, like DV.

If the field order looks wrong then it probably is. Reverse it (either in my AviSynth script or in your encoder) and it should look fine.

[Ed Cread]

Thanks for the help Kevin and Dan.

>2. use the parameter "OutputBFF=true" in the AVS script, if you are using Dan's advanced script. I tried both ways (upper/lower), and it only looks excellent if you use this parameter with BFF=true.

I ran the advanced script in VirtualDub -- I have the hd2sd.avsi file in the AVISynth plugin directory and other than what was already in the basic script sample from Dan, I only copied in this additional script into the premiere.avs file: hd2sd("debugtest4.avi", OutputFieldRate=60, OutputBFF=false, NR=5, Sharpness=0.5, VerticalBlur=0.5)

I am getting this error when I tell Virtual Dub to run the premiere.avs script:
"AVISynth open failure: script error: invalid arguments to function "isRGB" (hd2sd.avsi, line 52).....*\premiere.avs, line 16).

Cant seem to get the advanced script to work!

Also, regarding field order, if we are specifying the outputbff=true parameter, should I comment-out this command: "SelectEvery(4,1,2)" ?

[_Kevin_Haga_]

Ed, I got this error in one of Dan's previous versions of hd2sd.avsi - which he has since corrected. I suggest redownloading the latest script from his guide page. I downloaded it Thursday night (he updated it on Wednesday, 7/2/08 with this fix) and used it successfully.

[_videoad_]

Big problem, PLEASE HELP!!!

I'm editing an XDCam 1440x1080 file and everthing is good, BUT when I export the file with the Media Encoder I have try all the presets but there is no preset what showing me a fullscreen 16:9 1440 movie.

What preset or settings must I use to give me an 16:9 fullscreen output???

Please HELP!!!

Erik

[Dan Isaacs]

> I downloaded it Thursday night (he updated it on Wednesday, 7/2/08 with this fix) and used it successfully.

Yep. If something doesn't work right, always check that you've got the latest version of the script. I try to test things as thoroughly as possible, but mistakes happen: Fortunately, there are now a good number of others who are you using these scripts so I'm more likely to find and fix problems!

On another note, I noticed somebody from Adobe was browsing my example page on Premiere's interlaced scaling issues: It was apparently from a link in an email. Let's hope that means that some of the Adobe folks are at least discussing these problems internally.

[Ed Cread]

OK - the newer script fixed the error - thanks again for quick and helpful replies.

Dan - nice job on this. I wasn't a believer that it would be worth it to run these 3rd party plugs, scripts, etc., but it does make a visible difference.

Here is one part of the workflow I'm not sure I understand though -- I am using ProCoder 3 to do my encoding to m2v for DVD. When I try and open the premiere.avs script it gives me errors, but when I open the avi file that I have created via VirtualDub (by "save as *.avi" from VDub) it appears I am getting the end product as intended. Can one of you confirm I am essentially doing the same thing this way?

One thing that seems out of sorts is that it is only taking about 10 minutes for VirtualDub to save out my avi file (2 min video saving as 3.4GB avi). Kevin mentioned more like one hour per minute so am I not getting the full benefit here. My PC is dual core AMD 3000+ with 4GB Ram - nothing special.

Thanks!

Ed

[Dan Isaacs]

> I wasn't a believer ... but it does make a visible difference

I know. It's always that way! Nobody here ever believes me (boo hoo). The difference is huge.

> When I try and open the premiere.avs script it gives me errors...

I haven't used ProCoder in years. It supposedly supports .avs input. Can you please tell me the exact error? Which "version" of my script are you using: the basic one or the hd2sd() function?

In any case, saving the AVI from VirtualDub should be the same, except that there's an unnecessary RGB conversion -- not that it really makes much difference.

Perhaps ProCoder wants RGB? If you are using the basic script, try adding the line ConvertToRGB24() at the end of the script. Or, pass the parameter OutputColorSpace="RGB24" to the hd2sd() function in the advanced workflow.

> One thing that seems out of sorts is that it is only taking about 10 minutes for VirtualDub to save out my avi file (2 min video saving as 3.4GB avi).

Great! This also depends on the complexity of your project and a million other factors. Saving to an intermediary .AVI requires more disk space, but can greatly speed up encoding -- especially on the 2nd pass of 2-pass encoding. Also, Kevin is using CCE. Perhaps ProCoder is faster, I don't know.

Glad to hear of your success with the workflow!

[Dan Isaacs]

Hey all --

The advanced script has been updated again. Important corrections have been made to the script examples in the .txt file.

I also added the option of integrated 29.97i -> 23.976p conversion (set OutputFieldRate=24, SmoothTime=false)

Also, if you are encoding with CCE, ProCoder or writing an intermediary .AVI be sure to set OutputColorSpace="YUY2"...

...or possibly OutputColorSpace="RGB24" -- but only if you need to specify RGB output. (for use with TMPEG, for example)

[Ed Cread]

>I haven't used ProCoder in years. It supposedly supports .avs input. Can you please tell me the exact error? Which "version" of my script are you using: the basic one or the hd2sd() function?

The error reads: AviSource: Could not decompress frame 0
(E:\*myfilepath*\premiere.avs, line 1)

I'm using the hd2sd()script -- I think?!? I've selected some script commands you had in your text file and added to what was originally the 'basic' avs file I created from your basic workflow. Of course nothing would work then until I placed your hd2sd.avsi file into the plugin directory of AviSynth.

>Or, pass the parameter OutputColorSpace="RGB24" to the hd2sd() function in the advanced workflow.

Perhaps I am running before I've learned to walk, but you lose me with that one. What do you mean by "pass the parameter"? I tried adding the command you suggested but perhaps it was done incorrectly as I still get the same error.

If not out of line in the forum, I will copy and paste my script file from my 'premiere.avs' file that I am trying to open in Procoder and maybe you can see my obvious error? debubtest3.avi is my signpost file created by DEBUGMODE FS from premiere**

[Here is the script - less only the comments marked with #]

>avisource("E:\Video Projects\tests\AVISynth tests\debugtest3.avi")
ColorMatrix(mode="Rec.709->Rec.601", clamp=false)
LeakKernelBob(order=1, threshold=10, sharp=true, twoway=false)
Lanczos4Resize(720,480)
SeparateFields()
SelectEvery(4,1,2)
Weave()
Limiter(16, 235, 16, 240)
hd2sd(OutputColorSpace="RGB24")
hd2sd("debugtest3.avi", LimitWhite=false, OutputColorSpace="YUY2", Output601=true)
hd2sd("debugtest3.avi", OutputFieldRate=30, SmoothTime=false)

Thanks again for the assistance.

Ed

[Dan Isaacs]

Hi Ed --

Try this for ProCoder, upper field first (using the "advanced" version, your script only needs one line):

hd2sd("c:/path_to/signpost.avi", OutputColorSpace="YUY2")

That should work. Output will be 29.97i (TFF), YUY2 colorspace if your source is 29.97i.

In the future, please email me the script instead of posting here so as not to confuse others or clog up the thread. After it's all working for you, you may want to post the script you used, your MPEG2 encoder settings, etc.

[Ed Cread]

Hi Dan,

I added ONLY the line you provided and Procoder worked! Nice results and it only took about 10 minutes to encode the 1:51 second video.

I hate to wear out my welcome with more questions, but I am still a little confused on what you were meaning by telling me to do 'upper field first' in your latest post? I already had the script to do UFF [(SelectEvery(4,1,2)] and I have UFF selected in my export settings in Premiere. Are you suggesting something else?

Thanks very much for all of your time and assistance. I owe you an apology for thinking this was all a colossal over-kill considering my HD-SD videos already seemed to be looking good. It's the fine lines in the video that really suffer in the transition without doing this avisynth. I will enjoy the better outputs and it really wasn't that hard to get going with this workflow.

Ed

[_Kevin_Haga_]

Ed, are you using 2-pass VBR for your Procoder? Are you feeding directly from PP3 -> DebugModeFrameserver -> Procoder? or are you creating an intermediate file from VirtualDub before having it go to Procoder?

It's taking me a good hour to encode each minute of video, and I'm not sure that its the encoder that's slowing me down. I'm using PP3 --> DebugModeFrameserver --> CCE Basic encoder.

When I encode with AME (Adobe-only workflow), it's much, much quicker - however not the results I want... I'm running on a new Quad core and it's not being utilized.

[Dan Isaacs]

Just so we're clear on this, Ed:

> I added ONLY the line you provided ... I already had the script to do UFF [(SelectEvery(4,1,2)]...

If you are using the hd2sd() function, your .avs file should consist of EXACTLY ONE LINE (as quoted in my last response). Delete all the other statements and leave only the hs2sd(...) command. The script is a "one liner", so to speak.

> and I have UFF selected in my export settings in Premiere

As you should. HDV is TFF but DVD can be either. The hd2sd() function uses TFF by default. OutputBFF=true can be set to reverse the field order, which seems to be better when encoding with Adobe Encore.

CCE and other encoders make it easy to specify the field order of the source, so it doesn't really matter which you use as long as your encoder is aware of it.

[Dan Isaacs]

Hi Kevin ---

> I'm running on a new Quad core and it's not being utilized

It won't be -- as least when outputing via AviSynth. AviSynth is not a multi-threaded app (there is an MT version of, but it requires a different approach to take advantage of it -- which most of the existing plugins won't benefit from anyway.)

For maximum encoding speed (especially for longer projects and 2-pass encoding) I suggest writing an intermediary .AVI file from VirtualDub. This will allow CCE or ProCoder to use whatever resources they need to comrpess the file.

Also take a look at HC Encoder -- the freeware MPEG2 encoder I recommended on my workflow page. I've always been pleased by its results and I'm curious how it stacks up against ProCoder and CCE, as I've never really done a "head to head" comparison.

HC Encoder also has an option to automatically write a lossless file (not an AVI, but a proprietary format) on the first pass which it then encodes from on the next pass.

[Ed Cread]

>Ed, are you using 2-pass VBR for your Procoder?

Yes - I do 2 pass VBR with a minimum bit rate of 7000, avg. bit rate of 7500, and max @ 7900.

>Are you feeding directly from PP3 -> DebugModeFrameserver -> Procoder? or are you creating an intermediate file from VirtualDub before having it go to Procoder?

This is where I was having trouble understanding what exactly I was supposed to do. When I run debugmodeframeserver as my export from Premiere, it gives me the signpost file. I was then taking my script and opening it up in VirtualDub to scrub the footage. At first, I was putting out an intermediary avi file (about 3.4GB in size for 1 min 51 second video). It took about 10 minutes to encode.

Then, I tried opening up my avs script directly in Procoder, and it worked. Took about the same amount of time as it did to encode the intermediate file. **But now it seems my procoder is not happy with my avs script even though I made no changes? Working on that right now...

Ed

[Ed Cread]

>If you are using the hd2sd() function, your .avs file should consist of EXACTLY ONE LINE (as quoted in my last response). Delete all the other statements and leave only the hs2sd(...) command. The script is a "one liner", so to speak.

Aahhhhh....I thought you stacked up different scripts in the avs file (the one you suggested naming as premiere.avs in your guide) to pile in all the settings you wanted, such as color correction, sharpening, interlaced or progressive, etc.

So now my avs file is only one line, and the hd2sd.avsi is located in my avisynth directory. Can I make various selections, such as adding the "OutputBFF=true" WITHIN the hd2sd.avsi file versus doing so in the premiere.avs file?

I'm so close to getting it figured out but now I don't understand where I make my choices on the various options such as interlaced or progressive, TFF or BFF, etc.

Ed

[Dan Isaacs]

Ed -- Please read the .txt file that I provide in the hd2sd.zip file. There are plently of script examples. In your case (frameserving to ProCoder, BFF), a script like this should do the trick:

hd2sd("c:/path_to/signpost.avi", OutputColorSpace="YUY2", OutputBFF=true)

That's it. Use one line and pass the parameters you want to the hd2sd() function, as documented in the .txt file. And as for...
> Can I make various selections ... WITHIN the hd2sd.avsi file versus doing so in the premiere.avs file?

The file hd2sd.avsi defines the function. It's a "script", but it acts more like a plugin. Don't edit it. Just create a new .avs file in the same folder as your "signpost.avi" and paste the line above into it.

> interlaced or progressive, TFF or BFF, etc.

If you are going to DVD then chances are you want interlaced (unless it is 24p). You can choose TFF or BFF at will. DVD supports both. Just make sure that your settings in ProCoder reflect this choice.

[Ed Cread]

Thanks Dan - your last post gets me over that hump. Although I had read the txt file, it just didn't click with me about passing the different parameters into that one line.

Ed

[Dan Isaacs]

Just to post a minor variation of this workflow for folks with Cineform or Matrox systems... a little background first:

As I documented in another thread, PPro fails to make a distiction between BT.709 (the HD color standard) and BT.601 (SD). The long-and-short of it is that PPro "pretends" that everything is 601.

My script(s) fix this problem for HDV footage by performing a 709->601 matrix conversion. However, there is still the problem of RGB graphics being converted to YUV using the 601 coefficients. So... outputting via DebugMode in YUY2 will output the video colors correctly, but not RGB graphics. Outputting as RGB will screw up the colors of the video, but the graphics will be correct!

It seems to me that the first option is better and that is why I suggest frameserving as YUY2.

Cineform and Matrox have a way around this problem by performing their own YUV->RGB conversion on the HD video footage (so long as the captures are in their .AVI containers anyway).

If you are using a Cineform or Matrox project you should be able to get better color representation (and faster performance?) by bypassing the DebugMode FrameServer and outputting to an HD AVI file. You can then open that file through AviSynth just as you would the frameserver "signpost.avi" and passing it thorugh my script for downconversion.

[_chr_kel_]

Dan quick question, I have a project that I'm shooting on the HVX at 720p/30fps. I'm editing in PP and will go out to DVD. I guess since it's not interlaced I don't need to deinterlace but I still need to resize and convert for DVD. Would the above steps still work for that?

[Dan Isaacs]

Hi Chr --

How are your results using AME for 720p->DVD? Chances are they're fairly good, but I've never tried it.

You can likely increase the quality a bit with AviSynth, however (more accurate color, sharper scaling, etc.). Try this (instead of my "basic" script) on the frameserver's YUY2 output:

AviSource("signpost.avi")
ColorMatrix(mode="Rec.709->Rec.601", clamp=false)
Lanczos4Resize(704,480)
AddBorders(8,0,8,0)
# uncomment the next line for HC Encoder or QuEnc
# ConvertToYV12(interlaced=true)

[Dan Isaacs]

Long-time followers of these threads will recall that I used SmoothDeinterlacer in earlier versions of the workflow. After I became aware of a few issues, I switched to LeakKernel.

I've now realized this was a bad choice -- both because LeakKernel has its own issues and that I found a workaround for most of the problems I was having with SmoothDeinterlacer.

The "basic" workflow page and the "advanced" hd2sd.zip have both been updated, so please (re)visit the link in the original post.

[_Abdessamad_IDRISSI_]

hello Dan and thanx for the workflow.

I got a cannon xlh1 captured footage to premier under this specs;

Image Size: 1440 x 1080
Pixel Depth: 1440
Frame Rate: 29.97
Pixel Aspect Ratio: HD Anamorphic 1080 (1.333)
Fields order : upper field first
Display format : 30 fps drop frame TC

Then exported video to frameserver-->Virtual dub--> save as uncompressed AVI --> imported into encore.

I noticed some (bugs) concerning the border added to the video when viewed on Encore or exported to DVD ; is it normal? or should I remove out from script the line that says;
AddBorders(8,0,8,0)

Thank you anyway.

[_Abdessamad_IDRISSI_]

another bug with avisynth file when opened in virtual dub;

ther's no function named (colormatrix)

this happens even i copied the dll's to appropriate directory.. I have to try several times befor it works !!

[Dan Isaacs]

Hi Abdessamad --

I don't understand... In your first message it seems that everything worked fine and THEN you got an error about ColorMatrix?

> noticed some (bugs) concerning the border added to the video

It's no bug. 1440x1080 -> 704x480 (or 704x576) is a very accurate conversion of the apsect ratio. AddBorders pads this to 720. PPro's "native" export adds 5 pixel borders to each side (1440 -> 710), but this is not so precise. If you want to fill the screen (at expense of slight stretching), change the resize line to 720x480 (or 720x576 for PAL) and comment-out the AddBorders line.

I just copied/pasted the script I posted and it works fine. Be sure that the colormatrix.dll file is in the root of your AviSynth plugins folder.

> I have to try several times befor it works !!

No idea. I've never heard of that issue before. The only thing I can think of to make sure the plugin folder is referenced in your registry. More info here: http://avisynth.org/AviSynthPlugins