HDV --> SD DVD Workflow?

[hhansard]

Dont want to sound stupid or naive, but probably going to come across that way

I have shot [underwater] dv for years, and moved to hdv in 06. Since I have an end-to-end blu-ray setup with a broadcast scalier, my hdv footage looks great at home.

I was asked to take some of my hdv footage and burn an sd dvd for distribution. I took an edit of 1440x1080 and exported in PP-CS3 [movie] [Sorenson] as 720x480dv, no recompression highest quality (26G for a 30minute vid), etc, etc.. and imported it into a new PP3 d1 project. Burned the DVD again highest quality

I looked at the results on an sd 4:3 monitor terrible much worse then my old native dv footage. I expected to loose quality, but assumed that the scaling algorithm would smooth, and I would end-up with something acceptable --- but it does not look like it it looks like pixels are just dropped with no interpellation at all. I mean if you never saw the original footage, you might let it pass, but having seen the original footage, you can tell that the compression has killed it. I know that this is like a 2 (maybe 3) generation dupe, but I have access to the original pixels and would have assumed with minimal recompression/expansion the results would be as good as 1gen dv but I cant seem to get there

Can anyone point me to a good workflow [or some settings] to take hdv footage and cut a decent quality sd dvd using the production suite??

Thanks in advance,

Hugh

[_hconant_]

Dan,
I'm still watching. Thanks for sticking with it.
Howell

[hhansard]

This post will need to be broken-up due to size:

How did this start?
What are the issues?
Thread, Phase I
Thread, Phase II
What footage are we taking?
Who is the audience and what are their expectations?
What monitoring equipment is being used? Is it calibrated?
Does an all-Adobe workflow really exist?
What should Adobe do?

------------------------------------------------------------------
How did this Start?

Ive been shooting nature and underwater SD for years. Used all-Adobe workflow for output. No problems.

Started shooting HDV to BD with a combo of PP and 3rd party tools, then switched to CS3 and all-Adobe workflow.

Was asked to take some of my HDV footage and push for SD-DVD for distribution the BD was fine, so I figure cake. The raw tape is fine on scope and monitor.

Pushed the footage out and reviewed the SD output on a calibrated scope and monitor.

Houston we have a problem.

HDV  SD scaled push much worse native HDV

However, I shot identical footage in native SD and the output was much better then the HDV->SD scale/push.

I read all the documentation, on-line White-Paper PDFs, etc. Tried all combo of settings and still no-joy

Called adobe techs they verified my settings, then they went into deny deny deny then the techs claimed it was my footage

So decided to start this thread I figured that someone must know how to do this

I think some summary is needed. I can not be sure, but it appears that 50 or so messages have been deleted from this thread. Unfortunately, some of the experimental data and intermediate workflows appear to me missing. I will not be repeating them here. If they are really gone, then I will attempt to resurrect the best workflows.

--------------------------------------------------------------
What are the Issues?

1) Color Shift: This has been verified using a calibrated scope. On the original tape pushed to a calibrated broadcast output train, the color is perfect scope and monitor. After processing by a PP-only workflow, the color is shifted as verified on a monitor/scope and calibrated output train. The complexities of YUV -> RGB -> YUV color model mapping have been covered in other threads. From my perspective, I got correct color mapping when I shot SD and now I do not, despite verifying broadcast safe output. I recognize that there are differences in the limits for luminance and chrominance and they do not map exactly between D1, HDV and DVD, but these were all verified at a broadcast studio. There are complete articles and threads on these issues.

2) Field/Frame Artifacts: There are obvious interlace artifacts and haloing. The raw HDV tapes were compared to the PP-only workflow using calibrated broadcast equipment. When comparing the PP-only HDV->SD workflow directly to SD-only workflow, there is no comparison. There are threads and technical articles on scaling interlaced footage. The footage must be processed to a full-frame un-compressed format, and then sent through a scalar. There are about a ½ dozen de-interlace algorithms, so your results will vary. Obviously, you must re-interlace if you want to go broadcast. Scaling algorithm: The scaling algorithm used is critical. At this time, I am unaware that Adobe has published its scaling algorithms for PP nor weather it de-interlaces prior to scaling (and if so, how?)

3) Aspect ratio conversion: Objects of known aspect ratio do not appear to be correctly converted upon scaling. This was verified on calibrated broadcast equipment.

------------------------------------------------------------------
HDV-> SD Thread Phase I

A few things should be made clear and may be difficult if the postings in this thread have indeed been deleted: I spent years shooting nearly identical footage in native SD and that output was much better then the HDV->SD scale/push using the all-Adobe workflow. This was using multiple capture devices. The only commonality was the Adobe workflow.

I know a workflow exists I have friends who make documentaries in HDV for PBS, the BBC, the big Screen and broadcast. I have seen the results they are there. So I reached-out. Got access to a broadcast studio and pushed-out an SD through pro equipment. Results were fine all issues resolved. Why cant I get a quality result from an Adobe-based workflow?

Thread Phase I asked several questions:

Why are the issues not discussed in the documentation?
Why does Adobe tech support not recognize, nor admit to issues?
Despite monitoring and editing this thread, Adobe has never published a workflow.
Despite publishing all settings, footage, broadcast equipment used, and that a correct workflow has been published, Adobe employees have claimed that the problem may be challenging footage (i.e. we only can process wedding/corporate events/etc.) and in no way admits to problems with PP.

These questions were never addressed. Full stop.

------------------------------------------------------------------
HDV-> SD Thread Phase II

Several folks stepped to the plate and developed workflows incorporating third party tools. Dan is in many ways the driving force. He has started several threads trying to help the PP community. I have tried his workflow with good results. He is striving to make the 3rd party tools more accessible to the larger PP community. He should be supported.

Unfortunately, the bulk of the information from this point went private e-mail. I received more then 200 e-mails.

[hhansard]

What Footage are we Shooting? What are your Goals?
Weddings?
Events?
Training?
Products?
Broadcast?
Nature footage for the BBC?
Distribution?
Big Screen?

One of the early private e-mails from Adobe employees started talking about challenging footage. No definition of challenging was ever offered. But basically, the e-mail implied, if you are using color or contrast separation to manage negative space or have frames with high-color contrast rapidly moving against a contrast background (think BBC), then PP will have a problem.

Many folks evaluate their results based on their goals. When I have nature documentary footage for distribution (which is how this started) and I needed 50K copies, then that is my yard-stick.

------------------------------------------------------------------
What are the Expectations of the Audience?

The expectations of the audience and how they review the results differ greatly:
Wedding couple on a $300 TV?
Corporate folks projected at 1280x1024?
Training vid on a computer?
Broadcast needs to be broadcast legal?
Nature footage for the BBC?
Big Screen

When we consider the evaluation of quality, we need to consider the expectations of the audience.


------------------------------------------------------------------
How do we Monitor Output?

1) A consumer:
$89 DVD player to a $350 monitor?
2) Pro: Calibrated broadcast output train:
$6K VTR through a $17K Scalier, $15K scope to a $22K monitor?
3) Something in-between?

Obviously, there is a big difference. I really dont want to start ranting, but when I investigated many of the e-mail comments that this thread spawned, I found that over 90% of the output equipment was consumer-grade. Of course they could not see color shift they could not see calibrated color a scope, what is that? end rant

------------------------------------------------------------------
Does an all-Adobe Workflow Really Exist?

Depends on your customers expectations and how critically they evaluate the results. It depends on the footage. The more complex the footage, the larger the hill to climb.

------------------------------------------------------------------
Moving forward

Why cant Adobe simply admit that there are issues in going from HDV SD. I have been calling Adobe, writing in forums, surfing for answers and running my own experiments for almost a year and no-one can seem to provide an Adobe-based workflow that actually works.for challenging footage and no-one from Adobe will admit to a problem.

1) Produce a white paper to help your customers get the best results that they can.
2) Commit to improve the HD(v)  SD workflow in the adobe suite. CS4 is in Beta, I hope some improvements have been made.

I have been fortunate that I can get my footage processed in a studio. I think the PP community is fortunate that Dan is working on making freeware more accessible. But Adobe should step-up and get the turn-key solution working for all broadcast-legal footage not just the easy stuff or change their marketing literature accordingly with CS4 As I have offered repeatedly, I have drawers of HDVCAM tape that I would be happy to send

[Bob_Ramage]

Frankly, all the Adobe engineers need to do is shoot some basic HDV 60i footage then transcode to SD to see how bad the end result is. Dan's workflow is simple and very effective, but I agree it would be nice to see Adobe engineer a solution to this themselves.

[Dan Isaacs]

Hugh... That's the "wrap up" I've been waiting for! Nicely stated. Howell... Glad you're still on board. Nice to hear from you guys.

Thanks to all who have given me support in this and other "contentious" related issues (color conversions, PAR, etc). It is thanks to Hugh that I really, really got into this in the first place. I had my own methods that I used for my needs, but he inspired me make these more "accessible" and to share them with the PPro community.

"Challenging footage". Hah! The only way that is relevant is in that some footage is hard to deinterlace properly (especially for doublerate deinterlacing). Since Adobe's deinterlacer is as crude as they come, all footage presents an equal challenge and it will always provide the worst possible result. I submit that it is their software that is "challenged" :)

[Jeff Bellune]

>As alluded to in another thread, you may also find that some simple modifications to this workflow could achieve very nice podcasts, which may be of interest to you.

Dan,

What did you have in mind? Resizing my tutorials for podcast is done dynamically in After Effects, and the source/project/export from Premiere is progressive from the get-go.

It's possible that client videos may benefit from both the deinterlacing and the resizing algorithms in your script; is that the focus of your comments?

-Jeff

[Dan Isaacs]

Yes. Here's a modification of my basic script that will output to 424x240p (almost 16x9) square pixels at half the original framerate. This is assuming 1080i YUY2 output from DebugMode FrameServer (should work fine for both 60i and 50i input):

AviSource("c:/path_to/frameserver.avi")
ColorMatrix(mode="Rec.709->Rec.601", clamp=false)
ConvertToRGB32()
SmoothDeinterlace(tff=true, doublerate=false, lacethresh=24, staticthresh=35, staticavg=50, edgethresh=35, blend=true, showlace=false)
ChangeFPS(FramerateNumerator() / 2, FramerateDenominator())
Lanczos4Resize(424,240)

You can change the size to anything you like (that's Lanczos4Resize line). You can comment-out ChangeFPS(...) to maintain the original framerate. Also try setting SmoothDeinterlace -> "blend=false" for temporally sharper deinterlacing... Infinite options. Tell me what you're looking for and I'll tell you how to get it :)

[Jeff Bellune]

>Tell me what you're looking for and I'll tell you how to get it

I think I've got everything I need from the script. I'll be doing tests on 4x3 and 16x9 material.

What I do need now is a definitive source to enlighten me if the latest generation of "Classic" video iPods have native 320x240 resolution, or if Apple has managed to put 640x480 pixels into the same screen space. I'm off to do some research.

[Jeff Bellune]

That was easy. It's still 320x240 per the Apple web site. ;)

[Steven L. Gotz]

I actually own a Video iPod if you need assistance.

[Jeff Bellune]

Thanks for the offer, Steven. I have 3, including a Touch. ;)

I wanted to know the specs so that I could create appropriately-sized content before the whole import-into-iTunes-then-sync-your-iPod dance.

I didn't save the tech specs on the units (if they even existed in the packaging) so I needed a definitive reference. Apple's web site conveniently has that info. :)

[Steven L. Gotz]

I don't have the cable to play the video from my iPod to my television, but I believe that the point of loading larger video is to play it out at 640X480 to the TV.

I haven't checked to see if larger video looks better at the smaller size because everything I have put on my iPod looks great.

[Jeff Bellune]

>I believe that the point of loading larger video is to play it out at 640X480 to the TV.

That, and the fact that if you want to play the video in iTunes on your computer, 640x480 is much more watchable than 320x240.

I'm about to upload a couple of test videos with lower thirds that started life as DV footage, so I should have a good test result for Dan's AviSynth workflow versus just frameserving to Squeeze 5. Preliminary results using the QuickTime player have been interesting.

EDIT:

It's likely that in the near future I will need 4 deliverables for clients:

1. DVD
2. Flash for the web
3. iPod Large
4. iPod Small

I see my rates for deliverables going up... :)

[Steven L. Gotz]

I have found that most of the videos I designed for HDV or DV viewing are not appropriately edited for iPod.

I end up creating a 320X240 project and pan/scan to get the best part of each larger frame into the iPods smaller frame. I do the same thing with the videos I create for video contests that require small frames.

I edit first for my big HDTV, then I edit again for iPod/contest.

I often find that lower thirds become "lower halves" to be readable.

[Jeff Bellune]

>I often find that lower thirds become "lower halves" to be readable.

I'm deliberately testing LTs purely to judge the quality of difficult material.

[_Valera_]

Mr.Dan, what would be the right script to go from progressive 1920x1080 16:9 to SD 16:9. Also from interlaced 16:9 to SD 16:9.
Also will I lose any quality going that route as far as due to no 16:9 in avisynth?

thank you so much.

[Dan Isaacs]

Hi Valera ---

I posted back to the other thread, as it is more relevant there: http://www.adobeforums.com/webx/.59b5b2fb/51

[Jeff Bellune]

Dan,

My tests indicate that the SmoothDeinterlace function and the Lanczos4 resize function in AviSynth do not produce as good a result as the Smart Deinterlace plugin and the Lanczos3 resize algorithm in VirtualDub when going from interlaced SD DV to progressive iPod or Flash video. At least not without many, many hours of experimenting with the settings. I got superior results using Smart Deinterlace and it only took me 3 mouse clicks.

Do you have any canned settings for the SmoothDeinterlace function that you like to use for SD interlaced to SD-or-smaller progressive? The defaults you listed in your script, and the different settings that I've tried produce inferior results compared to doing it all in VirtualDub. Note that I did change tff=true to tff=false in the script before I started testing.

We could start a new thread or take this offline, whatever you'd like.

-Jeff

[Dan Isaacs]

Hi Jeff -- things can be a little different with SD -> low-res in a variety of ways... In these cases it may be easier to stick with VirtualDub, but I'll try to see if I can get you the quality you're looking for. (check your email)

[SiriNeosBoss]

Congratulations to Dan for your Workflow.
I have a HV30 Canon (PAL) and last weekend I recorded a lot during a 3 days trek in the Pirineos Mountains (Spain). Yesterday I followed the wokflow and the result was fantastic. Thanks Dan.

I would like you to know if the resizing why do you recomend resizing to 704x576 for PAL SD video edition.

In my opinion, I could understand the resizing from 1440x1080 to 704x480 for NTSC video edition, because the ratio
"Vertical Transformation" / "Horizontal Transformation" is equal to 1.1, good for good transformation. (the ration is not so good if you were transforming to 720x480 (1.125).

But, when you are resizing from 1440x1080 to 704x576 as you propose for PAL SD edition, the ratio is equal to 0,91666666. If the resizing is to 720x576 the transformation ratio is 0,9375, which is a better and there the image is 16:9.

Is there any reason to choose 704x576 for PAL edition instead 720x576, when you are having worse transformation ratio and you are loosing the origina proprtion (16:9)?

Sign: SiriNeos

[Dan Isaacs]

702x576 is the "internal" 16:9 (or 4:3) area of PAL SD content. I round this to 704, which is an even multiple of 16 (MPEG stndard).

Here is an interesting guide from BBC on this subject:
http://www.bbc.co.uk/commissioning/tvbranding/picturesize.shtml

[SiriNeosBoss]

Thanks Dan for your answer.

In the advanced script you can choose between 704x576 plus 2 vertical pillars of 8 pixels size each, or 720x? with two horizontal croping.
In the first option the image is reduced horizontally but the 704 are enlarged to 720 in the TV, so the image in the TV is ok. Am I right?

But in the second option, there are two pillars at the sides of the image of 8 pixels of size each (9 as is indicated in your link)that will dissapear in the TV. In this case you are missing part of the image (horizontal and vertical). Am I right?

In which cases is useful to use each option? How to decide?

It depends if I see the DVD in TV or in the computer?

[_Simon_Gilmour_]

Hi Dan!

I spent several weeks editing together a HDV project in Premiere, all looked great, then I came to try and export it to DVD - couldn't believe how poor the result was!! Blurry and out of focus. I have barely moved from my computer, even to sleep, for the past week trying to find a solution. I kept coming across your methods in the forums, but because it sounded so complicated, looked for others first. I have now tried every technique that has ever been suggested, but still not been happy. I became so disillusioned I found myself squinting whilst watching trying to fool myself into thinking it looked better - all to no avail. So I finally decided it was worth pursuing your techniques. At so many stages did I nearly give up when it just didn't seem to be coming together, and then finally it all slotted into place. I waited for the dvd to burn and nervously took it through to the player in the lounge. Blown away! An incredible improvement! Movement is beautifully smooth, and there's that detail I've been missing! Thankyou so much, saved my skin!

Simon

[Dan Isaacs]

No problem, Simon. I'm always glad to hear it 🙂 The process wasn't as complicated as you thought, was it? Now you've got the solution you can plug it in anytime you need to.

[Dan Isaacs]

Hey SirNeos,

Sorry, I must of have missed you post somehow until now.

The SD 16x9 aspect ratio is a little more "squeezed in" horiznontally as compared to HD 16x9. There are 3 ways to deal with this:

1.) you can stretch it about 3% horizontally to fill the frame (slightly distorted aspect)
2.) you can crop 12 pixels off of the top and bottom of the HD source, then scale it to fill the frame (correct aspect)
3.) pillarbox 8 pixels on each side of the output (correct aspect)

If you want 100% fullscreen playback on a 16x9 LCD computer screen, use #1 or #2. If you care about correct aspect, use #2 or #3. If you want to preserve every last pixel on the top and bottom of the original, use #1 or #3. For normal TV/DVD, any option is valid. I usually prefer the second option.

The choice is yours, of course :)