Encore always transcodes my x264 compressed files

[Eric_Daoust]

Hi everyone. I'm about to go crazy working on my first Blu-Ray disc! I work on a Mac Pro using Encore CS6 on Mac OS X 10.6.8. I am looking to use an AVC/H.264 encoded file by the x264 compression engine, which is supposed to be the best out there. For this, I have tried using FFmpegX and Handbrake with regular settings (1920x1080p, 23,976fps, 16:9, constant quality) starting from an Uncompressed 10-bit output from my Premiere timeline.

I am very satisfied by the compressed H264 outpout file, but no matter which application I use and the different configurations (tried different profiles inside Handbrake), everytime I import the  file, Encore sees it as a file that has to be Transcoded again. And of course you can imagine how the result looks like when an h264 file is encoded twice...

I have tried reading every possible forum on the subject of H264 Blu-Ray compatible files... without any success. I desperately need your help!

I have even tried pointing to the "encoded" file on my hard drive by telling Encore it has been encoded already, but the compilation of my Blu-Ray fails everytime when I does this.

I have succeeded once to create a file with Handbrake that is "sometimes" accepted in Encore (happened twice, I think), but it's bitrate was too risky to my taste for compatibility (38mbps) and therefore was only a test. I have no idea why this one worked...

[Eric_Daoust]

What's fascinating is that PMK3's sample works great. Can't wait to hear if he is using x264...

I've also made a few tests with a compiled version of x264 in terminalon mac, but it still says "not encoded" in Encore.

[PMK3]

Eric_Daoust wrote:

Can't wait to hear if he is using x264...

I'm using the Mainconcept Reference 2.2 encoder. Sony DVD Architect Pro 5 doesn't like the x264 encoder.

[Eric_Daoust]

Damnit. It seems at least that there is a mac version... Is the quality comparable to x264? I doubt it.

Anybody had ANY success using an x264 encoded file with Encore?

[tjh65]

Yes. My main problem was with amorphic video as encore would not recognise the correct sar. It turns out that Encore has not kept up todate with the H.264 specs for aspect_ratio_idc. I was able to patch x264 to fix this. I am still having problems with getting the interlace field order right but that doesn't seem to stop me building blu-rays. I would advise you to run x264 directly and not through a gui at the moment. These are my settings:

x264.exe  --level 4.1 --bluray-compat --tune film --bitrate 20000 --threads 16 --keyint 25 --open-gop --weightp 0 --slices 4 --vbv-bufsize 30000 --vbv-maxrate 25000 --tff --input-res 1440x1080 --videoformat pal --fps 25/1 --force-cfr --output-csp i420 --input-csp i420  --sar 4:3 --log-level info --output "output.264" "input.avs"

The unfixed x264 will work with progressive video but you will need to edit the options. I am using YV12 input. I am not saying these settings are optimum but they work with my HDV video. Always check the logs for warnings.

[Eric_Daoust]

What do you mean by "fixing" x264? I will try something close to your proposed settings and report on monday. Thanks!

[tjh65]

Look at this:

http://mailman.videolan.org/pipermail/x264-devel/2011-January/008186.html

Encore uses a table which has not been updated to the latest h264 spec. All you have to do is find the table in x264 binary and patch with a hex editor. You need to change 04 03 0E to 00 00 ff. x264 will then use the extended form of aspect_ratio_idc. Make sure you find the correct place in the binary by checking for the whole table. This will make x264 confirm to the old h264 standard. To summarize x264 will create an aspect_ratio_idc of 14 for HDV video which encore will not recognize.

The Mainconcept encoders gets the pic_struct wrong in the SEI. It uses 1 and 2 for top and bottom fields respectivly wheras it should be 3 and 4 as used by x264. Unfortunatly Encore perpetuates Mainconcepts bug and interprets pic_struct wrongly. To change to this in x286 would require the source to be edited but I don't think it is worth doing as the resulting stream will not conform to H264 standards. Encore will accept x286 video as don't transcode so encore will build a disc with H264 complient video which certainly not true using Mainconcepts encoder.

Please note that the values 1 and 2 should be used for paff interlacing and 3 and 4 for mbaff. Unfortunatly Mainconcept uses 1 and 2 for both. Encore thinks x286 video is progressive for tff video and tff for bff video.

Encore is very out of date and has numerous bugs which have been around for years.

Hope this helps

Regards Trevor

[Eric_Daoust]

Thanks Trevor. Unfortunately, I'm a real newbie when it comes to programming knowledge. I've used an Hex editor in the past on PC, but are you aware if there is a solution like this on the mac platrform which I'm currently using?

If it was just to patch the exe file with your recommendation, I could probably do it easily. But then again, I'll need access to a PC

You're saying also that any file encoded in Mainconcept won't be *really* H264 compliant even if Encore is accepting it?

[tjh65]

You only need the patch if you are using amorphic video ie non square pixels such as 1440x1080i25.

The interlace problem is seperate. There are 2 ways to encode interlace in the h264 specification PAFF and MBAFF. In PAFF pict_struct can have a value of 1 or 2. With MBAFF pict_struct can have a value of 3 or 4. Mainconcept encoders have got this wrong and use 1 & 2 for both PAFF and MBAFF. In other words Mainconcept does not conform to the standard for MBAFF encoded video. In general MBAFF works best and x264 only supports MBAFF.

Progressive video should be no problem as long as the basic rules are followed see

http://www.x264bluray.com/home

One importent point progessive video is not supported with HD at 25 or 30 fps. To get round this there is a special option in x264 called fake_interlaced which will turn progressive into fake interlaced complient with Blu-ray.

Hope this helps.

[Eric_Daoust]

It does help alot. I really appreciate your comments. I've tried compressing my file with these settings on my mac in terminal :

x264 --level 4.1 --bluray-compat --tune film --keyint 24 --open-gop --slices 4 --bitrate 35000 --preset veryslow --vbv-bufsize 30000 --vbv-maxrate 25000 --colorprim bt709 --transfer bt709 --colormatrix bt709 --fps 24000/1001 --force-cfr --input-res 1920x1080 input.mov --videoformat ntsc --sar 16:9 --output OUTPUT_tjh65.264

The program seems to work great and does not make any error message during or after compression. But when I import the resulting file in Encore, it still says that Encore has to transcode it and it seems that the content itself of the .264 file is corrupted. Here's a sample image (the video looks like this anywhere on the timeline):

[Eric_Daoust]

You can download a video sample here if you want to try and see the original specs of the video. Thanks in advance!

[Stan Jones]

Bad link.

[Eric_Daoust]

You're right. The original uncompressed sample is like 4gb... I'll see if I can take a smaller excerpt of just a few seconds for a test.

[Eric_Daoust]

Here it is!

[Eric_Daoust]

Anyone?

[PMK3]

x264 --bitrate 25000 --preset veryslow --tune film --bluray-compat --fps 24000/1001 --force-cfr --bframes 3 --ref 4 --no-weightb --weightp 0 --b-pyramid none --vbv-maxrate 30000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --min-keyint 1 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 -o out.264 input.file

Can you encode a 1920x1080p23.976 with the settings above and try it in encore. If this doesn't work then try by replacing --bluray-compat with --aud and --nal-hrd vbr.

[tjh65]

The minimum requirement is shown here

http://www.x264bluray.com/home/1080i

I have just built a 18gB blu-ray ok. I found that encore was very slugish with a .264 file. I multiplexed with tsmuxer to a m2ts file and it was alot better and did not transcode.

By the way I assume we are all talking about CS6.

[Eric_Daoust]

Thanks guys! You're giving me hope! One good thing:

@PMK3: I'm finally getting "Do not transcode" in Encore CS6 with your recommended settings. BUT, the video still looks like this when I encode it (even though there are no error messages!?!):

I've installed x264 following some online tutorials on the mac. I'm not experienced at all in compiling. It seems to work fine in Terminal when I encode, but the output is just wrong as you can see...

But we seem to have the right settings for Encore!! We're almost there I guess... What do you guys think about this strange output?

[PMK3]

Try this, use the settings that worked and add --muxer raw.

[Eric_Daoust]

Unfortunately, it's the same. Buggy video accepted as "Do not transcode" by Encore CS6. What's strange is that the duration of the resulting file (as read by Encore) is twice the length of the original uncompressed 10-bit file.

[tjh65]

Ah!! 10 bit. X264 does support high10 so try adding --profile high. You could try --level 4.1 aswell. Does the mac x264 support mov files then.

Encore and blu-ray only support 8bit.

[PMK3]

I agree with tjh65 that using 10 bit can be the reason why it's not working. The highest profile and level that can be used on blu-ray is high and level 4.1, blu-ray also only uses 8 bit and 4:2:0. I don't know if you are also using 4:2:2, so try with a source file that is 8 bit and 4:2:0.

[Eric_Daoust]

You guys are great. We are not there yet, but thanks again for your help. I have added --profile high (--level 4.1 was already there). When outputting Uncompressed 8-bit 4:2:2, I am getting something a bit more figurative (we can almost see the horse haha!) and pink this time. Duration is still a bit "longer" than the original file, but less than the Uncompressed 10-bit. Go figure.

Should I try the Animation codec or something else as the input file?

Thing is: I cannot export 4:2:0 uncompressed from Premiere in 8-bit. My only option is: Uncompressed 8-bit 4:2:2, which gives me:

Hints are still welcome. I feel we are close to the goal

[Eric_Daoust]

Settings:

x264 --bitrate 25000 --preset veryslow --tune film --bluray-compat --fps 24000/1001 --force-cfr --bframes 3 --ref 4 --muxer raw --no-weightb --weightp 0 --b-pyramid none --vbv-maxrate 30000 --vbv-bufsize 30000 --level 4.1 --profile high --keyint 24 --min-keyint 1 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 -o out_8bit_422.264 --input-res 1920x1080 input_8-bit_422.mov

[tjh65]

x264 should convert 4.2.2 to 4.2.0. Well it does on windows. You should be able to see the conversion mentioned in the x264 log. There are couple of options you could try --output-csp i420 --input-csp i422 if these are right for your system. I use avisynth to do the conversion 4.2.2 -> 4.2.0 but I don't think you can do that with mac.

[PMK3]

I wonder if it would work if we used another codec when exporting to quicktime. There is a codec called DNxHD that can be downloaded from here: http://avid.force.com/pkb/articles/en_US/Download/en423319?retURL=%2Farticles%2Fen_US%2FDownload%2Fe...

This codec is also available for windows, so this codec can be usefull when working with both MAC and PC. This codec is only available when exporting to quicktime, but I don't think that this is something to complain about when using a coded that is freeware.