HDV --> SD DVD Workflow (DebugMode FrameServer, AviSynth)

Report · Jul 02, 2008

While the old thread contains a wealth of great information, I've decided to break away from it. It's too old, too long, too confusing... In summary:

Many users are aware that Premiere/AME/Encore does a "less than stellar" job of converting 1080i HDV to interlaced SD DVD. I (and others) have tried every conceivable combination of options and found none that provided "professional quality" conversions. This is largely due to the way that Premiere handles scaling of interlaced material ... Example #1 ... Example #2 ... and Premiere's failure to convert from the HDV color matrix (Rec.709) to the SD specification (Rec.601).

In an effort to solve these problems, I developed a workflow (using several third-party freeware tools) that achieves results far superior to that of Premiere and/or Encore by themselves.

I have written a guide for this workflow, including step-by-step instructions and links to download all the required tools.

This page also contains a link to a more advanced option for doing these conversions (using mostly the same tools). Please do not attempt the advanced version unless you have already implemented my "basic" workflow!

The page and linked files will be updated frequently. It is a work in progress, but should already provide excellent quality. I look forward to comments and suggestions from this community (as always).

Enjoy! -- Dan

Report · Jun 18, 2009

Glad to hear things are working out for you. Actually, I made another patch to sd2hd() a few minutes ago, so download it again.

Report · Jun 18, 2009

I tried but just get zips with hd2sd.avsi from 16.06.2009. Is this the correct one?

I now know why the roofs in distance looked "less noisy" in VDub - they are simply not that sharp/detailed I get the same effect when using DeintMethod=2 and/or SmoothTime=true. One looses lots of details with that settings. I will keep that in mind for daylight footage where it is more important to get good mpeg2-compression. For my display footage where I have lots of black sky (=good compression) I will use DeintMethod=1+SmoothTime=false only to get maximum detail.

Slowly, slowly I understand and am REALLY happy with the results...

Report · Jun 18, 2009

Thanks again. As I'm sure you know, I have spent a lot of time on these workflows. Converting one format to another can never be "perfect" but I try as hard as possible to get the best quality while maintaining reasonable performance.

Again, SmoothTime=true is good for minimal motion stuff. It does not add blur on static area; only where there is motion.

In a few days I plan to release some greatly revised versions. I am also hoping to find the time to create some HTML documentation of the functions and complete end-to-end workflows for CS4.

The .zip you downloaded should be correct… note that the sd2hd.avsi file is newer.

Report · Jun 21, 2009

Hello Dan,

in the mean time I did lots of tries and I feel really convenient with the solution. The original 1920x1080/50i looks untouched very sharp and good, the 1280x720/25p from your workflow is so well deinterlaced and scaled that it looks even better than nativly filmed in this resolution, same with the anamorphic 720x576/25p. All these files are perfect sharp and clear without jaggies, artefacts or smoothing when saved as AVI out from VDub (opening your scripts with different parameters in VDub).

But I am not 100% satisfied with the 576/50i (TFF or BFF) from the script. I know, one should only judge it on a good CRT, but there is no CRT here. What I can clearly see is that there are much (!) more jaggies and stepping in the 576/50i as in the 576/25p after I let TMPGenc do the interlacing into 50i TFF. So, there is a large difference and I would have to use the 576/25p as source since it just looks so much better on TFT and plasma. But I would like to take your 50i, since it comes from 50p and the moving objects should look better.

Have you any comment about this? Yes, CRT would give probably a better image, but the difference between 576/25p interlaced in TMPGenc and your 576/50i is obvious.

But in total it is just overwhelming, thank you so much again.

Report · Jun 21, 2009

But I am not 100% satisfied with the 576/50i (TFF or BFF) from the script. I know, one should only judge it on a good CRT, but there is no CRT here.

As you said, the only way to evaluate 576i is on an interlaced PAL television. That's just the way it is. Many HDTVs, etc. are HORRIBLE at deinterlacing and upscaling… and you cannot evaluate interlaced quality on a computer screen.

more jaggies and stepping in the 576/50i as in the 576/25p after I let TMPGenc do the interlacing into 50i TFF

Because, in the later case, there is no real "interlacing". I'll explain:

DVD uses the YV12 (4:2:0) colorspace. In this mode, the chroma (color) resolution of PAL video is 360x288, or half of the original. There are two different ways you can encode 4:2:0, which are progressive and interlaced. In progressive YV12, color information is shared between neighboring lines of the 720x576 output – while in interlaced YV12, chroma is "combed" -- shared by lines 0+2, 1+3, 2+4, etc.

So, by feeding a 720x576/25p source to TMPG and choosing "interlaced" it is simply spatially interlacing the chroma channels. The luma channel is still 100% progressive. I believe this is actually the recommened way of encoding progressive PAL content – for compatibiity with older DVD players. I have seen many (clearly "progressive") PAL film titles on DVD that are encoded as interlaced.

Report · Jun 22, 2009

I love your explainations, Dan. Although I've read tons of forums threads in the last years and especially in the last months such things are not obvious, when one does not know to look for it

So, your script does real interlacing from 50p to 50i where TMPGenc would just interlace the chroma channels?!
This means: "your" 50i would be the more correct interlacing and additionally would deliver better moving objects, since it is not based on 25p - is this correct?

On the other side "my" 50i with TMPGenc based on 25p represent 100% valid "interlaced" streams, even for old standalone players, but indeed contains progressive material for the cost of worse movements because of 25p. Is this correct?

Is there any other aspect one should keep in mind? Eg. regarding the real "visible" resolution or sharpness?

Since today more and more people watch DVDs on flat TV or on PC-TFT monitors I tend to use the 25p because it simply looks better there... Mmmh.?!?

Report · Jun 22, 2009

So, your script does real interlacing from 50p to 50i where TMPGenc would just interlace the chroma channels?!

Correct. If you simply feed a 25p source to TMPG (or most any encoder) and choose an "interlaced" encoding mode, then this is what you'll get.

This means: "your" 50i would be the more correct interlacing and additionally would deliver better moving objects, since it is not based on 25p - is this correct?

Yes.

Is there any other aspect one should keep in mind? Eg. regarding the real "visible" resolution or sharpness?

Not that I can think of. Using any of the deinterlacers in hd2sd() should provide good enough results. DeintMethod=1 is "best," while DeintMethod=2 is much faster and may provide results that are just as good for this purpose with most material. Avoid oversharpening during this process. It can be tempting to try to "tease-out" lots of detail by using the Sharpness parameter; but you will also increase artefacts and make the stream more difficult to compress. On the opposite end, hd2sd()'s noise reduction option can improve compressibility and often make the picture much nicer – but care must be taken not to obscure subtle details.

Since today more and more people watch DVDs on flat TV or on PC-TFT monitors I tend to use the 25p because it simply looks better there... Mmmh.?!?

Not a bad philosophy at all. I only wish that older DVD player were able to handle the "true" progressive 25p and 30p.

Another option is to do it the "film way" (but in reverse, for your situation). Most PAL DVD players can handle 24p NTSC. So, assuming you don't mind the pitch shift in the audio (-4%), you can slow it down to 24p:

# get source and deinterlace to 25p

AviSource("lagarith_1080i50.avi")

ConditionalSmoothBob(DeintMethod=1, OutputColorSpace="YUY2")

SelectEven()

# slow it down by 4%

ar = Round(audiorate())

AssumeFPS(24000, 1001, sync_audio=true)

# convert to 720x480 (24p NTSC)

hd2sd(OutputColorSpace="RGB24")

# restore audio samplerate

Try { SSRC(ar) }

Catch(err_msg1) {

Try { ResampleAudio(ar*2).SSRC(ar) }

Catch(err_msg2) {
ResampleAudio(ar) } }

Report · Jun 18, 2009

Ufff, this is an option I would have never thought about.

Thank you for you concrete guide. But - how one gets more information out from a lower resolution? Probably you will get a nice upscaled video and with the sharpening it looks a little better. But in the end you probably won't have a better picture than upscaling with a very good DVD-player or TV?!

But I will give it a try, it indeed could be very useful to avoid mixing of different resolutions. I have some footage which is filmed in 1280x720/50p which has to be combined later with 1080i footage for Blu-ray production. So you solved a problem before I asked about it

BTW, another thought - what is your or others experience about the following:

when deinterlacing 1080/50i to eg. 1080/25p do you see less stuttering of fast movements than with filming in 1080/25p directly? You add motion blur while deinterlacing, so you get "smoother" moving objects for the costs of loosing sharpness/resolution - but will this prohibit fast objects from stuttering at 25p?

Report · Jun 18, 2009

First: I made another little fix to my hd2sd + sd2hd package that will improve the sharpening quality when upscaling SD footage to interlaced HD.

But - how one gets more information out from a lower resolution?

The same way you get 50p from 50i : interpolation. What this means is that the software looks at pixel A which is next to pixel B and calculates, "if there were a pixel between A and B called 'x', what would 'x' look like?"

Now, some interpolation is good or bad, fast or slow… EEDI2 is slow and very good. Unlike normal "scaling", it does an exhaustive search for the best solution – trying its damnedest to preserve continuous diagonals across A, x and B.

Here is an example I made a long time ago to show the difference (in a very simplified manner) between various upscaling methods:

EEDI2 does a pretty nice job; even at 400% of original size.

But in the end you probably won't have a better picture than upscaling with a very good DVD-player or TV?!

If you know of any upscaling DVD player that currently has technlogy like this, then please let me know! Hell, Photoshop doesn't even have technology like this: that's Photoshop's "best" (bicubic) above the EEDI2 image.

when deinterlacing 1080/50i to eg. 1080/25p do you see less stuttering
of fast movements than with filming in 1080/25p directly?

You woin't have any less stuttering (unless you perform blending or some other pseudo-motion blur post-processing). I think the best way is probably to film 25p with a shutter speed of 1/50th second.

Report · Jul 02, 2009

Dan, Thank you so much for a great workflow. I am trying to use your advance hd2sd but get lost. Is there a guide fo it somewhere?

Report · Jul 02, 2009

Well, what exactly are you stuck on? Tell me what you tried and what happened so I can help you.

Report · Jul 02, 2009

I have follow your basic workflow. Do i still use the "premiere.avs" script in HCEncoder, or do I use the following:

hd2sd("signpost.avi",
\                    OutputFieldRate = 60,
\                    OutputColorSpace = "YV12",
\                    NR=2,
\                    GrapeSmootherAmount=30,
\                    Sharpness = 0.5,
\                    VerticalBlur=0.25,
\                    ScaleBlack=true,
\                    ScaleWhite=true,
\                    FixWrongRGB=true
\)

for step 5b)

Also, do I have to wait for HCEncoder to finish encoding before importing the MPEG2 video into Encore? The process takes about 30hrs to complete encoding a 1hr10min sequence. Anyway to speed up the encoding process?

Report · Jul 02, 2009

If you are using HC Encode, be sure to set HC's Settings 1 tab to interlaced/TFF (upper fields first)

Set DebugMode Frameserver in YUY2 color mode
Export "signpost.avi"
I would recommend trying a more basic script like this first:

hd2sd("signpost.avi", NR=2, GrapesmootherAmount=30)

Of course, you can play with the other options also, but this is a better place to start.

For a faster encoding in HC Encoder, download and install the Lagarith lossless codec (if you haven't got the latest version of it already)

Open your .avs in VirtualDub. Go to Video / Compression and choose the Lagarith lossless codec. Be sure to go in the Lagarith codec options to ensure that multithreading is enabled.
select Video / Fast Recompress
Choose File / Save as .avi… we'll call it "lags_sd_yv12.avi" for this example.
Create another (very simple) script, call it "hc_source.avs":

AviSource("lags_sd_yv12.avi")
Open hc_source.avs with HC Enc and encode it to MPEG2. This will provide much better speed for two pass encoding… and you can always re-encode from the lossless AVI if you want to change your settings in HC (lower bitrate, etc.).

Report · Jul 03, 2009

Dan Isaacs wrote:
For a faster encoding in HC Encoder, download and install the Lagarith lossless codec (if you haven't got the latest version of it already)
Open your .avs in VirtualDub. Go to Video / Compression and choose the Lagarith lossless codec. Be sure to go in the Lagarith codec options to ensure that multithreading is enabled.
select Video / Fast Recompress
Choose File / Save as .avi… we'll call it "lags_sd_yv12.avi" for this example.

May I hook in, here?

I didn't even know that one can change the codec for the "Save as AVI..." option in VDub

Thanks a lot! What codec is used by default (I clicked to fast without checking before)?

For lossless codec and letting VDub doing no image processing "Fast Recompress" means "100% quality" though, am I right?

I do have a question to HCEncoder. I would like to use it but it crashes always at 2-3% encoding of second step. First VBR run is fine, then it crashes soon after beginning second run with

"../kmp_runtime.c", line 5235: OMP runtime internal error: assertion
failure.

Please submit a bug report with this message, compile and run commands used,
and machine configuration info including native compiler and operating
system
versions. Faster response will be obtained by including all program
sources.
Please send submissions to http://premier.intel.com/

on Win XP32 SP3 with i7 920 CPU. Do you have any idea - is this a problem with my CPU because if the intel-link?

And, Dan, you prefer HCEncoder over TMPGEnc. And earlier you asked if I would use Adobe Media Encoder. Do you have any suggestion about how good is the encoding of HCE compared to eg. TMPGEnc/AME? What I noticed is, that although TMPGEnc and AME use MainConcept encodings in H.264 looks with the standard settings of AME better with little less bitrate then in TMPGEnc.

I am still looking for best paramaters to encode in 1280x720/25p in H.264 for YouTube and for delivery on data-disk.

Sorry, this is OT, I will stop that here...

Report · Jul 06, 2009

I do have a question to HCEncoder. I would like to use it but it crashes always at 2-3% encoding of second step. First VBR run is fine, then it crashes soon after beginning second run with

Sorry. I have no clue about that one.

Report · Jul 06, 2009

I do have a question to HCEncoder. I would like to use it but it
crashes always at 2-3% encoding of second step. First VBR run is fine,
then it crashes soon after beginning second run with

...that was none of my questions, but tx anyway

Michael Weber

mo-mo productions

Michael Weber KG

Feldgasse 9

A-2100 Leobendorf

www.mo-mo.biz

+43 - (0)664/2153766

Report · Jul 06, 2009

that was none of my questions, but tx anyway

Yes. That was a reply to Kuro. I answered your questions in the preceding post.

Report · Jul 06, 2009

ok, the reply to Kuro just landed in my mailbox, sorry to bother you with

that. Tx very much for your support, i will check it out as soon as

possible...and will surely come up with more questions

Michael Weber

Report · Jul 07, 2009

Dear Dan, dear Kuro,

for some reason the whole correspondence between the two of you runs over my

mailbox...it started yesterday when i got 1 mail from dan answering my q and

a second from dan answering kuros q. can you sort that out please?....

ty again

Michael Weber

Report · Jul 07, 2009

Michael – Nethier Kuro nor I have any control over what messages you receive in your mailbox.

Report · Jul 04, 2009

Hello Dan,

I tried your workflow and have 1 problem with the result: when changing the aspect ratio to the giveb PAL resolution of 704,576 the output shows the original 16:9 picture squeezed into a 4:3 frame, which seems logical as the picture isnt cropped. so I recalculated and tried it with 704,396 which gave me a correct aspect ratio but an inferior picture quality in Encore. what's wrong here?

Michael (Vienna, Austria)

Report · Jul 06, 2009

Hi Michael ---

From your message I gather that you are using my basic workflow. I suggest you download the hd2sd package and use that instead, like:

hd2sd("signpost.avi", OutputColorSpace="YUY2", OutputBFF=true)

First of all, it allows you to make a very brief script (only one line in this example). The quality is also much better that the basic version and you have many more options (see the hd2sd.txt file included for more information).

Do not resize to 704x396. The correct way to us Interpret Footage in Encore and choose (in your case) PAL widescreen PAR.

Report · Jul 07, 2009

Dear Dan,

thank you again for spending so much time, especially in answering my questions here on this page. I really hope others will profit from reading it too... If one only could print this page "No. 6" - it does not seem to be possible.

But, I have one very special question, again.

As you maybe remember I film in 1920x1080/50i because I want to have highest possible resolution and need 50 frames because of fast full-screen motion (fireworks effects).
I want to convert the videos always to

- keep 1920x1080/50i (encoded in H.264/mp4 for BD-R with BDMV)
- 1280x720p (data-files for uploading in web and for displaying on PCs where Blu-ray is not possible)
- 720x576/50i for DVD
- 640x360p for small web videos

So far everything was clear to me, because of your massive support here.

But two days ago I regognized by chance, that H.264-mp4 files in 1280x720/25p won't play sharp on my 50'' Kuro Plasma. Probably this is because of the image processing of the TV. I would believe that a similar image processing will happen to a lot of TVs so I want to avoid those problems.

I then recognized that 50 frames in 1280x720p is displayed much, much sharper on my TV. It is so sharp that it almost "hurts the eyes" and its not easy to tell the difference to 1080/50i. This even happens when I do render with your hd2sd to 720/25p only and then encode in TMPGEnc as 50p (probably only frame doubling will happen there).

So, my question is: what is you suggestion please, how one should deinterlace/scale and encode to achieve best "720/50p" for fast motion.

I tried both,

1.) the normal conversion to 25p with SmoothTime=true (to have smoother movements by blending frames). Fast and easy, but not absolutly perfect movements and little softer because of blending frames (SmoothTime=false looks little sharper but has clearly visible stuttering)

2.) and interlacedOutput=false + OutputFieldRate=50 to have "real 50p". This one takes 12 hours for 20 minutes video on my i7-QuadCore, uauauhaaahhh!

In both cases I then render in TMPGenc with 50p in H.264/AVC as mp4 with MAIN profile and level 4.0.

Do you have other ideas or recommendations for getting best 720/50p out of 1080/50i? Earlier you wrote:

Here is my advice on that front: If you have a lot of full-screen motion (zooms, pans, car races, football games) then you definitely want to set SmoothTime=false for 50i->25p conversions. If you mostly have stuff that contains minimal motion (such as static tripod shots of interviews, etc.) then SmoothTime=false can be a good choice. In either case, you'll probably get the most natural result by shooting a 1/50" shutter speed.

there is a typo, probably you mean SmoothTime=true for full-screen motion?

Thank you so much and if it is enough, just say "no more", I will understand

Report · Jul 07, 2009

So, my question is: what is you suggestion please, how one should deinterlace/scale and encode to achieve best "720/50p" for fast motion.

OK. To create 720/50p from 1080/50i, simply do:

hd2sd(interlaced_out=false, OutputWidth=1280, OutputHeight=720)

Report · Jul 07, 2009

sorry to insist, but as you see in the text below, this message (which is

clearly meant for Kuro) was sent (from you) to smilingdolphin7 (which is

me), so I guess your mailserver somehow got in confusion with our

identities. As I stated b4, it started yesterday when you sent an answer to

me, followed 1 minute later by an answer to Kuro which also went to my

adress instead of his.

best wishes

Michael Weber

mo-mo productions

Michael Weber KG

Feldgasse 9

A-2100 Leobendorf

www.mo-mo.biz

+43 - (0)664/2153766

Adobe Community

HDV --> SD DVD Workflow (DebugMode FrameServer, AviSynth)