P: DNG Converter doesn't extract original raws from Denoise DNGs

[johnrellis]

Adobe DNG Converter doesn't extract the original raw embedded in the DNG created by Enhance > Denoise (run from with LR or CR). Tested on LR 12.3 / Adobe DNG Converter 15.3 / Mac OS 13.3.

1. Place this Nikon Z 9 raw into a folder by itself and import it into LR:

https://www.dropbox.com/s/6sew7rh7j08lsfv/DSC_1260.NEF?dl=0

2. Do Enhance > Denoise.

3. Move the generated DNG into a new folder by itself.

4. Run Adobe DNG Converter, click Extract, select the folder from step 3 as the input and output folders:

[Rikk Flohr: Photography]

I was watching the other thread, @johnrellis . Logged. 

[TheDigitalDog]

I can confirm this behavior. Tried on three different cameras and with/without proprietary raws and DNGs converted on import. 

To Adobe:

1. Is the DNG converter supposed to do this?

2. Why isn't such a function offered in Lightroom Classic? And Adobe Camera Raw. 

3. Do users even know that the original is embedded into the Enhanced DNG? Why not state this in the Enhanced dialog? 

4. Better still, offer a check box "Embed original into DNG" so we can disable this. I'm keeping the original; I may not want the original also embedded into the new Enhanced Linear DNG. 

[TheDigitalDog]

To add:

I just tried Ehnance to apply Raw Detal: no original embedded into the DNG. Same size as the original.

I then tried Super Resolution and again, the size is 3X, making be believe there is no original embedded into that new DNG. 

Adobe, this is a bit messy. I'm not sure about the 'concept' of embedding the original raw into (only?) Denoise was explained and there needs to be an option to not do so and of course, an option to remove that original. 

[johnrellis]

I drilled into some LR-produced DNGS using Exiftool, and I no longer think there's a bug in Adobe DNG Converter or in the Metadata panel.

When users ask why Enhance DNGs are so big, several experienced people in the forums are reporting that the Enhance DNGs "embed" the original raw. But there's ambiguity (and confusion) in the term "embed".

Enhance DNGs include both the linear raw data produced by Super Resolution, Raw Details, and Noise Reduction and the original raw mosaic (color filter array) data, converted to standard DNG representation.  So in that sense, the Enhance DNGs do "embed" the raw data.   But they don't include byte-for-byte copies of the original raw file.

DNGs produced by LR's Convert Photo To DNG with the Embed Original Raw File option include not only the mosaic data converted to standard DNG representation, but also an exact copy of the original raw file, stored in the DNG:OriginalRawImage field.  Enhanced DNGs do not contain that field.  So in this sense, Enhance DNGs do not "embed" the original raw file+.

I don't know why Enhance DNGs contain the original raw's mosaic data. It makes the file 25% larger, but it does allow you to run Denoise on Raw Details DNGs.  Perhaps most users would rather have Enhance DNGs 25% smaller and lose the ability to run Denoise on Raw Details DNGs.  Or perhaps there's another reason for keeping the mosaic data in the DNGs.

Regardless, I think we should stop saying that Enhance DNGs "embed" the original raw, saving "embed" for the "Embed Original Raw File" option that includes the byte-for-byte copy of the original raw file.  Instead, we should opt for precision and say that the Enhance DNGs include the original raw mosaic data. 

* * *

DNGs, like TIFFs more generally, contain multiple representations of an image. Here's a summary of the representations stored in an Enhance DNG, as reported by Exiftool.  (An IFD is an "image file directory", a subsection of a TIFF.)

[TheDigitalDog]

Good detective work.

To summarize: the original raw is not embedded into the new Linear DNG, a copy, converted to DNG, lossless, is embedded.

If my understanding is correct, we still need to be told this, ideally with an option not to, and we need a way to extract it if embedded.

I'd also like to know why this is only the case with Denoise when using the Enhanced dialog and the options.

[TheDigitalDog]

Here's what would be a welcome addition: 

 

[Jeff Schewe]

Not sure why anybody would think the Ehnace functiuon would actually embed the original raw file in a Linear DNG file...the ONLY way to create a DNG file with an embedded original raw file is to use DNG Converter and use tghe option to do so. Nowhere is it stated in ACR or Lr docs is there any hint that the original raw file is is involved is in Lr if you ask for the original rraw file and the DNG to be stacked.

[robert36972564]

I myself (LRc user, not experienced with the new enhancing features nor with DNG workflows) assumed so, because 1. the Lightroom settings offer this option and 2. the denoised DNG files are 4x as huge as the RAW files.

[johnrellis]

"Not sure why anybody would think the Enhance function would actually embed the original raw file in a Linear DNG file"

As I described above, I think the confusion arose from community experts posting that denoised DNGs "embedded" the raw data.  The use of "embedded" is ambiguous -- the DNGs do include the original raw's color filter array (mosaic) data, converted to standard DNG format, as well as the enhanced linear data. In any other context, that use of the word "embed" would be fine, but in this context, "embed" has long been used as the product label for including the original raw file byte-for-byte.  The denoised DNGs certainly do not include the original raw file.

[TheDigitalDog]

Jeff, the only reason some think that is because some Adobe folks told us that (a DNG copy of the original raw). Indeed, it is not stated in ACR or Lr docs is there any hint that the original raw file is involved. But some team members have stated outside such documents that is the case. And yesterday we were told this is not exclusive of Denoise in the Enhance dialog!

[Rikk Flohr: Photography]

This has been reviewed by the Camera Raw team and determined not to be a bug. I have reverted this to a discussion.

The intent of the mosaic data being included in the Denoised result is not to extract an original file at a later date. It is more accurate to say the original mosaic data is included for future iterations of the tool to accurately reprocess the original data for improved models and other possible enhancements (excuse the pun).  Ultimately, there is no additional 'file' within to extract. 

[johnrellis]

Great, thanks for confirming that.