HDV --> SD DVD Workflow?

[hhansard]

Dont want to sound stupid or naive, but probably going to come across that way

I have shot [underwater] dv for years, and moved to hdv in 06. Since I have an end-to-end blu-ray setup with a broadcast scalier, my hdv footage looks great at home.

I was asked to take some of my hdv footage and burn an sd dvd for distribution. I took an edit of 1440x1080 and exported in PP-CS3 [movie] [Sorenson] as 720x480dv, no recompression highest quality (26G for a 30minute vid), etc, etc.. and imported it into a new PP3 d1 project. Burned the DVD again highest quality

I looked at the results on an sd 4:3 monitor terrible much worse then my old native dv footage. I expected to loose quality, but assumed that the scaling algorithm would smooth, and I would end-up with something acceptable --- but it does not look like it it looks like pixels are just dropped with no interpellation at all. I mean if you never saw the original footage, you might let it pass, but having seen the original footage, you can tell that the compression has killed it. I know that this is like a 2 (maybe 3) generation dupe, but I have access to the original pixels and would have assumed with minimal recompression/expansion the results would be as good as 1gen dv but I cant seem to get there

Can anyone point me to a good workflow [or some settings] to take hdv footage and cut a decent quality sd dvd using the production suite??

Thanks in advance,

Hugh

[mickkeay]

I think I am gonna have to follow Dan too. I have a number of HDV projects that I burnt to blu-ray (or Quicktime for the web) - all great. This week I saw this thread and tried some SD DVD outputs myself. I tried the following:
HDV 1440 x1080i (PAL)project timelines

1. Allow PP to output to Encore and burn a SD DVD - terrible as expected
2. MPEG2 SD outputs (various) to burn with Encore, Nero, TMPGEnc - no improvement
3. MPEG2 HDV 1440 x 1080 from media encoder and tried in all the above dvd creators above - Not much difference. TMPGEnc was marginally better but still not acceptable (compared to old results I got with good old DV)
4. H264 Bluray out (big long wait....) and manually into Encore - still no better - very poor

So I looked at Cineform but the price puts me off (if indeed it would be a solution to this?)

So Dan - can you confirm (for an amateur like me) that the basic steps / workfow go like this:

Debugmode Frameserver (as a PP plugin) enables you to output the HDV timeline to a big avi file? Then after installing AviSynth (and SmoothDeinterlace plugin) I would run Virtualdub, drag my frameserved file onto it, load the avs script (in this topic) into VirtualDub - and run it! Now I am not quite sure where this leaves me? I then need CCE or QuEnc? I assume I end up with a DVD compliant MPEG2 somewhere

>you said....Just frameserve the full-quality frames from Premiere to a capable video processor (AviSynth) and MPEG2 converter (CCE, QuEnc, etc.) and you're done.

As you see, I am a tad confused as to the logical steps and where each bit of s/w fits in

[Dan Isaacs]

Hi Mick ---

No -- the FrameServer circumvents the need for a big .AVI file. It is an .avi, but it only "points" to the Premiere timeline and does not contain data for any frames. Export using the FrameServer plugin at full-res (1440 x 1080) in RGB mode.

VirtualDub is not really needed for this process, but you can drag your AviSynth script into VirtualDub to preview it and make sure you're on the right track before you start a long MPEG2 encoding process.

You'll need to adjust my AviSynth script for PAL, though. I think all you need to do is change the "BicubicResize" line to 720x576.

CCE or QuEnc are the MPEG2 compliant encoders. Use QuEnc: It's free.

[Dan Isaacs]

One quick follow-up regarding QuEnc:

QuEnc requires YV12 input, so add the following as the LAST line in your .avs script:

ConvertToYV12(matrix="rec709", interlaced=true)

[_Wesley_C_]

>No wonder my HDV to SD DVDs look like crap. I think I'm going to just shoot 16x9 SD for now until there is an easier solution.

Wow - this thread is amazing... As I said a the CS3 windows thread a short time ago, I can tell you with absolute certainty that my videos that are shot with my canon HV20, ingested with HDV split as M2t files and used in a premiere pro CS3 project with a 720x480 SD widescreen preset are noticably better in image quality (I'd say roughly 25% better if I had to gage it) than my prior videos shot in SD with my canon minidv camcorder.

I am sure to reverse the field dominance on the HDV clips and then export LFF using adobe media encoder with 2 VBR pass settings (min about 7.5MB and max about 8.0MB)

It works and I have had many many positive comments on my DVD output image quality.

[mickkeay]

Now that is interesting Jackie... I shall try that as soon as I can. Today I have been importing my HDV projects into a new SD project, and manually rescaling on new timelines and (whilst the results are better than letting CS3 go out to SD from an HDV project automatically) I have still not been overly impressed. I have not (yet) played with field stuff.

Did you try the reverse field dominance and LFF export directy from an HDV project by any chance?

[Guest]

>the FrameServer circumvents the need for a big .AVI file. It is an .avi, but it only "points" to the Premiere timeline and does not contain data for any frames.

I tried to install FrameServer, but never got it to show up in PPro CS3. I see that it has not been tested beyond PPro 1.5. Have you got it working in CS3, Dan?

Export "by reference" was a very handy capability within Quicktime and would like to get FrameServer working.

[Dan Isaacs]

Yeah, it works great in CS3 for me. I also had no issue in CS2.

If I remember right, you need to be careful about the Premiere plugin directory during install: it won't default to the right place.

Uninstall the frameserver plugin and reinstall it. Make sure after you install it that you can find the file:

cm-dfscPremiereOut.prm

located in...

\Program Files\Adobe\Adobe Premiere Pro CS3\Plug-ins\en_US

If not, uninstall it and try again.

-- Dan

[Andy Urtu]

Jackie,

On your workflow, when you bring in the HDV clip into a SD project, you state you don't scale. Once the clip is on the timeline, I assume you right click on the clip and choose FIELD OPTONS, then click the reverse dominance box.

When I use your workflow, If I did it right, I get really bad interlacing "jaggies"

Also, do you manually scale your clip to 45% or some other number?

Any more details you can give would be helpful.

Thanks.

[mickkeay]

I just tried Jackie's settings and although the quality looked good the end dvd video 'shimmered' - I guess as a result of the fieled dominance reversal. Getting there though.

On differebt note: (the frameserver plugin needs to go into the en_US plugin directory. The install wanted to default to C:\Program Files)!

[Dan Isaacs]

Seriously, folks... You will never get anything approaching "professional" results from 1080i -> DVD using only Premiere! It's just not possible. You can try all the different field settings in Premiere and that still won't crack it.

You need an adaptive deinterlacer and PPro ain't got one. There are probably some 3rd party plugins available for this, but you'll still lose all the field information: Outputting a true "interlaced" DVD would be impossible.

Premiere seems to use a very crude Bob / Weave scheme when scaling interlaced clips that ALWAYS yields shimmering and artifacts. The HDV signal is YV12 planar (4:2:0) whick means that color information is stored in 2x2 blocks. So if you simply "Bob" it into separate fields without proper interpolation then you've seriously compromised the color channels in your video.

You need to interpolate and/or motion-estimate the fields into full resolution frames at double the orginal frame rate prior to scaling and Weaving.

I'm not making this stuff up, y'know :)

[Harm_Millaard]

Dan,

Your info is valuable and much appreciated. There is one thing missing however and that is the fact that commercial DVD's are interlaced. I can't understand why people would want to convert interlaced material to progressive and then convert back to interlaced for recording on DVD.

Remember that progressive capable DVD set top boxes only came to market around 3 to 4 years ago and anything older than that can still play commercial DVD's, even though those older players are interlaced.

[hhansard]

Dan,

Sorry that I could not get back you your e-mails right away, I was in the city yesterday fooling around with some footage Nice to have friends with the right equipment 

I am trying to understand the limits of the NTSC performance envelope. What it really means to be broadcast legal and what are some of the problems that may ariseAnd get a bit of an education in the process Ill post results as soon as I can digest them

I am also having e-mail problems I tried to respond to your e-mails over the past 24-36 hrs, I cant send, only receive

Can you send me the script file that you mentioned

Is there any order / trick to install the new plugins? I know CS3-MS installs in a different place to account for PP and AE Anything beyond that?

I have a few sub-experiments in place to learn the new tools, but this slow-going

I do save some specific questions, but my last 3 emails were bounced When they get it working for me, Ill send them

Thanks,

Hugh

[Dan Isaacs]

Hi --- for Harm Millard:

I have not forgotton that most DVDs are interlaced. That's my whole point. It's simply a fact: when interlaced sources are scaled to a different size they are no longer interlaced, as such: The lines get streched or squeezed or blurred together and cannot be separated anymore.

That is why you must deinterlace before scaling. The method of deinterlacing is what's important here: If you want to recreate the interlaced signal, you must convert your 60i (or 50i) signal into 60p (or 50p) somehow. This is called "Bobbing".

Here's how it works (using 1080/60i -> NTSC as an example):

1080/60i -> (Bob) -> 1080/60p -> (Resize) -> 480/60p -> (Weave) -> 480/60i

As I said, simple Bob routines make for rather flickery video -- so you need a deinterlacer that does a more sophistacted "Bob".

[Dan Isaacs]

Hank --

Please email me when you can. Read this regarding the FrameServer plugin installation:

Dan Isaacs, "HDV --> SD DVD Workflow?" #105, 28 Feb 2008 8:08 pm

As for the AviSynth plugis, just copy the .dll files to the \AviSynth\Plugins folder.

-- Dan

[_ESC_in_KC_]

>On your workflow, when you bring in the HDV clip into a SD project, you state you don't scale. Once the clip is on the timeline, I assume you right click on the clip and choose FIELD OPTONS, then click the reverse dominance box. When I use your workflow, If I did it right, I get really bad interlacing "jaggies" Also, do you manually scale your clip to 45% or some other number?

Wanted to chime in on this workflow -- Yes, once the clip is in the timeline, right click it choose field options and then reverse dominance. The interlacing jaggies are on my computer monitor as well. Take it all the way out using the Adobe MEdia Encoder 2 VBR LFF as previously described, and burn it to a TV. It will look waaayyy better than you are viewing on your monitor. When I first saw those jaggies I was sick - but I saw somewhere to not fret over it and burn the DVD, which I did and it looks great - Try it!

Depending on the scene I also can get some shimmering (lots of repeating lines will do it every time). Try setting reduce flicker from same menu as the field dominance and it will reduce by about 80 to 100% depending on the scene.

>Did you try the reverse field dominance and LFF export directy from an HDV project by any chance?

Have not tried this - when I have worked in HDV project and output as SD DVD, it has looked just as good as the other work flow we are discussing. Only now I have learned that I prefer the HD in SD because of the flexibility of panning and zooming the footage with the same results in IQ.

Anxious to hear how it goes for you - burn the DVD and watch in on LCD or picture tube TV. You will be amazed at how much better it looks than on computer.

[_Corey_Vidal_]

I have kept silent, but I have been reading this thread every day for the past week or so. As such, Hugh, I'm wondering if you're getting to the point where you have found the absolute best method to provide best quality from HDV to DVD. When you're done, I'm almost thinking you should start a new thread saying "THIS IS THE BEST WAY", and provide an easy step-by-step format. Reading through 100+ messages can be a bit tedious. Or maybe you could just send me an email. :)

[_Joey_K_]

I've had excellent results by exporting the HDV edit to tape and then playing that tape in a sony HDV camcorder with the ilink conversion on. this makes clear clean SD.

for quick DVD's without a menu I play the HDV tape via firewire directly to a consumer DVD recorder (panasonic DVD recorders have an option to create a DVD which just plays the program without going to a menu).

it's a long way around but it works. if you want an HDV version of your program then it's not even an extra step.

the result definitely looks better than a DV-originated DVD.

[hhansard]

Joey: Thats a very interesting idea If I get this right, you import into a HDV project, do all your editing in HDV then let the output hardware do to the conversion to SD? So the output hardware scales, maps the color space and remaps the fields for you If I got that right, then that is cool.

Corey: you may be correct. After I get Dans workflow implemented, Ill do some side-by-side testing against my best effort work flow and some others suggested and simply post the best results to date. I think it is clear that there are some folks who use the turn-key PP/Encore workflow and get results that they are happy with, and there are some that are disappointed and looking for alternatives. At this point, I am simply looking at getting results that are worthy of broadcast and will adopt the first workflow that gets me there If I find one, then Ill write-it-up

[mickkeay]

Dan - Have been trying to get your workflow going. Got the frameserver plugin and already had avisynth 2.5 Still puzzled / being thick :-)

So with My HDV timeline I go file/export movie / chose the Debug frameserver chose RGB settings and destination. Now what happens? What do I do with the notepad created avs script? I was expecting to use VirtualdubMod to open the script and then drag the "target avi" from frameserver onto the input panel. But Virtual dub doesn't seem to support the avs script. Is there a clear step by step tutorial anywhere for hooking all this together - it would be real useful.
Regards to all - Mick

[Dan Isaacs]

Hi Mick --

There's an AviSynth tutorial in the docs and most AviSynth plugins come with their own documentation and examples.

VirtualDub (and VirtualDubMod) both support .avs files as input. Do you get an error when trying to open it? Check your AviSynth intallation. Also make sure you have the SmoothDeinterlace plugin installed in the AviSynth/plugins folder.

Make sure that the path to your "frameserver.avi" is in the first line of the script (ex: AviSource("d:/media/frameserver.avi")).

Remember --- VirtualDub does not output MPEG2. It is simply used (in this case) to preview your .avs file.

Download QuEnc or HCEncode (or buy CCE). These are MPEG2 encoders that accept .avs scripts as input.

I know I said you should output the FrameServer as RGB earlier, but I've become aware of a bug in Premiere when converting YUV <-> RGB.
(see http://www.adobeforums.com/webx/.3c0648b1 )

Therefore, I recommend YUY2 output from Premiere instead.

Also, you need to convert to YV12 for QuEnc or HCEncode. Here's an updated example script (assuming YUY2 / 1080i input and YV12 interlaced (upper field first) output for QuEnc):
------------------------------------------------------

avisource("d:/path_to/your_frameserver_file.avi")

# uncomment the next line if you want 4x3 instead of 16x9
# Crop (168, 0, -168, -0)

# uncomment the next line if your input is RGB
# converttoyuy2(matrix="pc.709").levels(0,1,255,16,235,coring = false)

SmoothDeinterlace(tff=true,doublerate=true,lacethresh=24,staticthresh=40,staticavg=85,edgethresh=40,blend=true,showlace=false)

bicubicresize(720,480)
SeparateFields()

# upper field first... use SelectEvery(4,1,2).Weave for lower
SelectEvery(4,0,3).Weave()

Limiter(16, 235, 16, 240)
ConvertToYV12(interlaced=true)

[mickkeay]

OK - thanks for your patience Dan.
Do I just need the dll in my avisyth plugins folder? There was a bunch of files in the zip and the readme didnt really say. Cheers Mick

Files:
SmoothDeinterlace.txt
SmoothDeinterlacer.rc

SmoothDeinterlacer15.h
resource.h
avisynth.h
SmoothDeinterlacer15.dsw

SmoothDeinterlacer15.dsp
SmoothDeinterlacer1.cpp
SmoothDeinterlacer.dll

[Dan Isaacs]

Yes. The other files are the C source code (if you should want to compile the .dll yourself)

[Dan Isaacs]

Mick, others...

Any luck with the workflow? There are tons of tweaks, etc. I can recommend to improve the basic AviSynth script I posted here, but I need to know if you've got it working yet as-is (and hopefully gotten some AviSynth basics under your belt along the way).

-- Dan

[mickkeay]

Hi Dan
Not quite (my day job has taken me away - Grrr) I am just experimenting with some basic Avisynth scripts so I can understand what I am doing (line by line) Hope to give it a go on a new HDV project tonight - Thanks again

[mickkeay]

Hi Dan
Not having much success. My first problem is that although SmoothDeinterlacer.dll is in my avisynth plugins folder, when i attempt to open the avs file (with QuEnc) it moans that smoothdeinterlace does not exist. (so for now I commented it out) Then QuEnc starts to encode and returns an immediate general (has to close) error. So I tried to open with VirtualDubMod. Frameserver is set to YUY2 as suggested but Virtualdub objects with a YUY error - "cannot decompress to a RGB format" So I changed the frameserver to RGB and commented out the - converttoyuy2(matrix="pc.709").levels(0,1,255,16,235,coring = false) - and then Virtualdub monitors the frameserving. I shall now try HCEncode maybe.
Thanks Mick