Creating DNG HDR from resized DNG from Fujifilm RAW files discards/loses the lens corrections

[Erik36545164gcm6]

I do real estate photography and a new work flow proces in my company has discovered a strange behaviour/bug in Lightroom Classic which causes a rather irritating time waste. I use Windows 11, LrC 14.3.1 and Camera Raw 17.3.1.

We shoot 3 brackets -3, 0, +3 - I use Fujifilm so I get Fujifilm raw fileformat RAF.

The proces is as follows for delivery of DNG-HDR to post production.

1. Convert the bracketed RAW files to DNG resized to 3000x2000px. The converted DNG files retains the lens correction. (this can be done in stand alone camera raw or through LrC - it makes no difference to the end result).

2. Import resized DNG to LrC, autostack and merge the resized DNG to DNG-HDR.

The problem now is that the DNG-HDR that is created, has lost the lens corrections and it cannot be (re)applied throught LrC. So there is vignetting and barrel distortion visible in the final.

My colleagues using Canon, Sony or Nikon do not experience this problem. I and my colleagues that use fujifilm all do.  Except when we create DNG-HDR files from full sized non converted Fujifilm raw files (but this is very time consuming - roughly 4x process time!)

Does anyone know what the problem is or how to prevent LrC discarding the lens correction in the DNG-HDR merge process?

[GoldingD]

Convert the bracketed RAW files to DNG 

Using what software? You sort of indicate you may be accomplishing that in camera, but also imply the possibility in LrC. Some Fuji X users absolutely hate Adobe Camera RAW around Fuji X images and might instead use Iridient X-Transformer.

The steps you are using are important to understanding what may be going wrong. Not all forms of DNG conversion follow the complete Adobe concepts for DNG (Adobe owns DNG), and some options in various methods might strip info.

resized to 3000x2000px

Probably not the cause of your issue, but why resize outside of LrC? Hard drive space constraints?

[Erik36545164gcm6]

As written in my post, I have done the proces with converting from RAF raw file to resized DNG with Adobe Camera Raw stand alone program or through LrC on windows. Same result - the created dng-hdr discard lens correction either way. 

I transfer the raw photo to computer and use Adobes programs for the entire process. 

I have tried Iridient and it actually works - when making dng hdr from dng made in Iridient the resulting dng hdr retains the lens correction, but, since I'm am using windows the Iridient app for Fuji it does not allow me to resize/change the dimensions of the original raw file. Only Iridient developer for Mac allows you to resize. So nothing is gained by using Iridient since I cannot resize.

And yes adobe for fuji is not my preference. All work that is not real estate work is processed in CaptureOne. 

[johnrellis]

"Convert the bracketed RAW files to DNG resized to 3000x2000px [in] LrC"

If we can provide a precise recipe for reproducing the problem, then we can file an actionable bug report (otherwise, Adobe won't pay much attention). To that end:

1. Which particular camera model(s) is exhibiting this problem?

2. How are you resizing the raw files down to a 3000 x 2000 DNG in LR?  The menu command Library > Convert Photo To DNG doesn't let you change the dimensions during conversion. Similarly, if you export the raw to DNG format, Export won't let you change its dimensions.  (Do you mean "crop" rather than "resize"?)

[Erik36545164gcm6]

My colleagues and I have identified the problem using Fujilfilm X-T5, X-T4, X-H2 and X-H2S.

I attach screenshots of the process of saving/converting the raw files to DNG files. Here done through bridge or LrC, but it's the same if I open the files in Adobe Camera Raw stand alone.

So after I transfer raw files from camera to computer these are the  steps.

Either:

1. Open files in Bridge and choose the raw files and  "Open in Camera Raw".

2. Save the files from camera raw to DNG format 6mb (3000x2000px) - so no cropping done.

Or

1a. Import raw files in LrC. Select raw files and choose Export.

2a. Export files resizing - Long edge 3000px.

Doing this either through Bridge or LrC does not make a difference for the final result of merging the DNG to and HDR.

3. I import the created DNG files in LrC. Here I select the brackets and choose "merge HDR".

When the DNG HDR has been created, the lens correction is lost and cannot be reaplied.

My colleagues using Sony, Canon or Nikon that follow the exact same steps do not experence the same problem.

Is the description clearly laid out?

[.Michael.]

@Erik36545164gcm6 : I have seen during my Fujifilm X-T2 and X-T5 HDR processing "often" the lose of the lens profile at the resulting HDR-DNG. But in difference to you I could re-apply the profile. Im using at LrC picture import the RAW to DNG translation without croped dimensions.

[JohanElzenga]

The vast majority of lens profiles is only available for raw files and linear RGB files like the DNG that Lightroom creates for HDR and panorama. It looks like that somewhere in the process, and the resize action seems the most likely moment, these files are converted to normal (non-linear) RGB files. That would be why the lens profiles are lost and cannot be reapplied. Because Fujifilm sensors are different than Bayer type sensors, I could imagine that this only happens with Fujifilm and not with other camera brands. 

[Erik36545164gcm6]

As the lens correction is still present in all of the reized DNG before the HDR merge, I would assume that it's that the HDR merge process that is the villain.

[Erik36545164gcm6]

Merging HDR from full size DNG instead of original RAW files does not give any gains.

The goal is to be able to merge the 3000x2000px DNG to DNG-HDR and have lens correction remaining.

It's no problem making DNG-HDR from orignal RAW files and have the lens correction still present - it just takes 4 times as along as with the resized image. A days work is 65-75 HDR merges. Without resizing this process wil be roughly 50 minutes compared to 12-13 min with resized files. So would really like to find a solution for this 🙂

[Jao vdL]

Can you share a few raw files that this happens to? People here (and perhaps some Adobe folks) can try and follow along and see what the issue is. Upload to a cloud service such as Dropbox and share the link.

[Erik36545164gcm6]

Here are one shot - 3 bracketed raw files for you to experiment with.

As written: Creating HDR from orignal RAF is no problem aside the time consumption.

Creating HDR from resized DNG discards lens correction even though it is present in reiszed DNG before HDR merge. This does not happen with Canon, Nikon or Sony resized dng's.

https://drive.google.com/drive/folders/1QMHw2d8maSF8E2RCcVaO4kF2tGhgfAg-?usp=sharing

[Jao vdL]

Thanks,

I tried to reproduce. I imported these three raf files. Exported them to lossy dng v16 with long side at 3000 pixels

to same folder with automatic reimporting them to the catalog. Hit create HDR on the new files and the result came in fully corrected with the lens correction set correctly as you can see in screenshot below

LR Classic 14.3.1 on MacBookPro M1 Max. 

This seems to work correctly on my machine at least with the release version of Classic.

Also tried this in Cloudy and it comes in fully corrected with the right profile:

No clue why this wouldn't work correctly on your computer. Perhaps a hardware dependent bug.

[Erik36545164gcm6]

OK. Thanks for trying that. My LrC also says the lens correction is applied but it really hasn't. So - are you sure the lens correction is applied correctly? Try flicking back and forth between one of the bracketed shot and the hdr. In the small snippets you have shown it seems that there is barrel distortion on the door and the lines/mortar in the tiling. Could you upload the final hdr and one of the resized dng brackets? 

[Jao vdL]

>My LrC also says the lens correction is applied but it really hasn't. So - are you sure the lens correction is applied correctly?

Turns out you are right. It is applying the lens correction incorrectly. I could only really see that by comparing the merge from the full res images to the merge from the reduced res images as you can see below (HDR gets blown out in screenshots. The window actually has all the detail when viewed on my display)

It is correcting but only partially which you can see by toggling the correction on and off. Sorry for missing this initially. This is clearly a bug indeed. Just directly merging to HDR is much faster for me but that might depend a lot on your hardware and might not be a solution for you.

[Jao vdL]

P.S. I timed what it takes to export, reimport and convert to HDR vs just creating HDR directly and it is faster to just create the HDR from the original files:

Reduce size by export to dng using a export preset-> automatic reimport -> merge to HDR: 35 seconds

Direct merge to HDR: 13 seconds

The actual process is faster for the direct merge but I counted the right click to get to the HDR dialog.

Now I tested this I can see that you are right that even though the interface says it is correcting for the lens distortion and vignette, it isn't actually doing that for the reduced size export image. It seems to only correct about 50% of the distortion! So there might be a bug here indeed. That said, just doing it from the orginal raf files is a lot faster anyway so that is what you should do. Interestingly if you merge to HDR using the raf files and then export to reduced size dng from there, the lens correction is correctly applied to the final file, so this is indeed some kind of effect from the merging to HDR using reduced-size dng files.

It will be even faster if you have a folder full of brackets and just select all -> enhance-> merge to HDR and it will automatically merge all HDR brackets.

[Erik36545164gcm6]

Thanks for investing time and effort and also identifying that something is not working as expected. I will try timing my workflow for different processes to see what is fastest. Although for me - hdr merge full sized takes 4 times longer than the reduced size. Resizing raf to dng only a couple of seconds pr file. 

[johnrellis · Accepted solution]

@Jao vdL, thanks for pointing out you need to select Lossy Compression in Export to reduce the size of the exported DNG.

@JohanElzenga, reduced-resolution DNGs created by Export and Photo Merge are both linear-raw DNGs (I was surprised by the former), and they have the full suite of lens profiles available.

@Erik36545164gcm6, I filed a bug report here:

https://community.adobe.com/t5/lightroom-classic-bugs/lens-corrections-don-t-work-with-hdrs-merged-f... 

I selected sample raws from five cameras in my test database: Canon EOS R6m2, Fujifilm X-T5,  Nikon Z 7_2, ODS OM-1, Sony ILCE-7C.  All the raws except those from the Canon have embedded lens profiles, while LR supplies the profile for the Canon.

All five cameras exhibited the same problem: HDRs merged from reduced-resolution DNGs didn't have lens corrections applied correctly. In addition, for the Nikon, the HDR merged from full-resolution DNGs also didn't have lens corrections applied correctly. 

Thus, this isn't specific to Fujifilm cameras.

[johnrellis · Accepted solution]

I timed merging HDRs from 15 sets of 3 Fujifilm X-T5 raws on my Macbook Pro M2 Max, which is pretty fast.

Method 1: Photo Merge > HDR applied to the raws.

Total time: 255 seconds.

Method 2: Export the raws to 3000-pixel long edge lossy DNGs, then Photo Merge > HDR applied to them.

Time to export: 47 seconds

Time to merge: 108 seconds

Total time: 155 seconds.

Thus merging the full-sized raws goes 1.6 times slower than producing and merging reduced-resolution DNGs. That is, the latter goes 39% faster.

[johnrellis]

Also, I wondered if applying Photo Merge > HDR to smart previews would have the same bug. That could be an easier way for you to achieve the same speedup in the workflow, since smart previews are just lossy, reduced-resolution DNGs 2880 pixels long edge.  

Unfortunately, smart previews have the same bug.

[Erik36545164gcm6]

Thanks for investigating this, and filing a bug report. My setup is not as fast as yours although I do have a Lenovo P1 workstation laptop. so not a sluggish laptop. 14 sets of 3 Fujifiln X-T5 raws takes just short of 9 min to merge. The equivalent for resized is around 2min 10s... So really a big difference for me 😕