P: Regarding JXL Compression in derived Merge/Enhance DNG Images

[tigre58]

Why with LR CC v.13, the files panorama files are now saved as lossy files?

[Rikk Flohr: Photography]

The compression method for derived DNGs has been changed from JPEG to JPEG-XL, which provides a smaller data footprint without loss of quality. 

Creating a merged DNG such as a panorama is already a “lossy” process because the merged pixel data has been demosaiced, aligned, and blended from the original photos.  Using JPEG XL compression makes a much smaller visual change.

[tigre58]

But , in previous versions of LR CC,  panorama files were saved as lossless files. Is posible change this set before making a panorama?

[Rikk Flohr: Photography]

See additional comments added above. 

[JohanElzenga · Accepted solution]

But , in previous versions of LR CC,  panorama files were saved as lossless files. Is posible change this set before making a panorama?

By @tigre58

Do not look at the words 'lossy' and 'lossless', look at your image and see if there is any visible effect of this new compression method. Recreate an older panorama and compare the old massive DNG with the new, much smaller DNG. Do you have any reason to be concerned?

[ClintGryke]

I just noticed this change today and came across this thread in search of the answer which is neatly provided thanks. However, it does raise a couple of questions. This may be pretty pedantic but it would be really interesting to know what the experts think.

So I've been experimenting with the new HDR Edit and Export feature in Lightroom. I have an Apple  XDR monitor which I have been using for HDR video up to now but this ability to edit and export stills in XDR is very exciting. In fact this is how I realised the Merge to HDR DNG file had changed as I was doing a HDR merge to test the HDR editing in Lightroom Develop Module.  Now in the guidance from Adobe (Edit and Export in HDR), it recommends using JPEG XL or AVIF files for export for sharing and web galleries and TIFF or PSD where additional HDR work is required.  The implication here is that the lossy compression of the merged DNG file is of no noticeable importance if not doing further editing but it may well be important if doing further editing.

So my question is in the case of merged DNG files, is there any point in exporting as lossless TIFF or PSD as against JPEG XL? In other words, is the data already lost in the same way as it would be if exporting an 8-Bit JPEG originally derived from a raw file for example as a TIFF or PSD? There is a huge difference in the file sizes for sure. My main usage at the moment would be importing into Final Cut Pro and some editing may be needed there to color match my HDR video.

[JohanElzenga]

Yes, the data are lost. The fundamental difference between lossless compression and lossy compression is that in lossless compression the decompressed image is bit for bit identical to the original. In lossy compression that is not the case. The decompressed file will not be absolutely identical to the original file. Whether that leads to any practical and visible differences is another matter that I cannot answer. My own suggestion is always that the proof of the pudding is in the eating, not in reading the recipe. Try it yourself.

[ClintGryke]

Sure - I can't see any difference from detailed examination of a few DNG files derived from the older and new processes. Perhaps with further editing? 

[reproo2773183]

The compression method for derived DNGs has been changed from JPEG to JPEG-XL, which provides a smaller data footprint without loss of quality. 

Creating a merged DNG such as a panorama is already a “lossy” process because the merged pixel data has been demosaiced, aligned, and blended from the original photos.  Using JPEG XL compression makes a much smaller visual change.

By @Rikk Flohr: Photography

Are you saying that previously a merged DNG Pano used an older JPEG encoding that supported 16bit?

[Rikk Flohr: Photography]

"The compression method for derived DNGs has been changed from JPEG to JPEG-XL, which provides a smaller data footprint without loss of quality. "

"Are you saying that previously a merged DNG Pano used an older JPEG encoding that supported 16bit?"
That appears to be exactly what was said. 

[reproo2773183]

Thanks Rikk, Crystal clear now 🙂

Panorama Merge in Photoshop then saved as Layered PSD, completely avoids Lossy compression? or same as the old Lightroom?

[JohanElzenga]

Panorama merge in Photoshop saved as layers is completely different. The compression depends on how (which file format and which compression) you decide to save it.

[reproo2773183]

Up to 5 minutes ago I thought that was the case :), not so sure anymore.

[JohanElzenga]

Up to 5 minutes ago I thought that was the case :), not so sure anymore.

By @reproo2773183

Why? You are comparing apples and not even oranges, but potatoes. Panorama merge in Lightroom creates a DNG that is auto saved. Panorama merge in Photoshop is done with RGB images rather than raw images, and there is no auto save to any mandatory file format. You choose the file format and the compression.

[reproo2773183]

love the anology.

in French

Pourquoi? Vous comparez des pommes et même pas des oranges, mais des pommes de terre.

So might be closer than you assume. 🙂 LOL

[madbrain76]

Sorry to jump in to this old thread. JPEG compression includes a lossless mode. Is that the compression scheme that LRc used prior to version 13 ?

JPEG-XL also features a lossless mode, for that matter. Why not use that ?

If LRc moved from lossless JPEG to lossy JPEG-XL, that does not seem like a good compromise to me. The user should be able to choose whether to prioritze the file size or preserving the entire data. I would personally prefer the later.

Tangentially, I would like to have the ability to do further operation on panorama and HDR DNGs, such as applying AI denoise, etc. That is currently not possible, even though could be applied to the original DNG files prior to photo merge.

[madbrain76]

Actually, it turns out the compression scheme for photo merge DNGs was changed from lossless Adobe Deflate to lossy JPEG-XL .

[madbrain76]

Rikk,

That statement would appear to be incorrect. At least according to exiftool, Photo Merge DNGs were compressed with Adobe Deflate in the past, not JPEG. And Adobe Deflate does support 16-bit per color channel.

[John R Ellis]

@madbrain76: "according to exiftool, Photo Merge DNGs were compressed with Adobe Deflate in the past, not JPEG."

I observe something different. I just tested LR versions 6.14, 9.1, 10.4, 11.4, 12.5, and 13.5.1.  In all except 13.5.1, Exiftool shows that Photo Merged DNGs were 16-bit with JPEG compression:

======== pano.6.14.dng
[System] File Size                  : 153 MB
[SubIFD] Subfile Type               : Full-resolution image
[SubIFD] Compression                : JPEG
[SubIFD] Bits Per Sample            : 16 16 16
[SubIFD] Photometric Interpretation : Linear Raw
======== pano.9.1.dng
[System] File Size                  : 152 MB
[SubIFD] Subfile Type               : Full-resolution image
[SubIFD] Compression                : JPEG
[SubIFD] Bits Per Sample            : 16 16 16
[SubIFD] Photometric Interpretation : Linear Raw
======== pano.10.4.dng
[System] File Size                  : 152 MB
[SubIFD] Subfile Type               : Full-resolution image
[SubIFD] Compression                : JPEG
[SubIFD] Bits Per Sample            : 16 16 16
[SubIFD] Photometric Interpretation : Linear Raw
======== pano.11.4.dng
[System] File Size                  : 152 MB
[SubIFD] Subfile Type               : Full-resolution image
[SubIFD] Compression                : JPEG
[SubIFD] Bits Per Sample            : 16 16 16
[SubIFD] Photometric Interpretation : Linear Raw
======== pano.12.5.dng
[System] File Size                  : 158 MB
[SubIFD] Subfile Type               : Full-resolution image
[SubIFD] Compression                : JPEG
[SubIFD] Bits Per Sample            : 16 16 16
[SubIFD] Photometric Interpretation : Linear Raw
======== pano.13.5.1.dng
[System] File Size                  : 27 MB
[SubIFD] Subfile Type               : Full-resolution image
[SubIFD] Compression                : JPEG XL
[SubIFD] Bits Per Sample            : 16 16 16
[SubIFD] Photometric Interpretation : Linear Raw

Exiftool isn't showing the actual parameters used for the compression schemes. Whatever parameters were used for JPEG compression weren't very lossy (if at all), considering the size of the DNGs.

You can download the raws and DNGs here:

https://www.dropbox.com/scl/fi/sr8cnkpllumfecz6gp68l/pano-dng.2024-09-25.zip?rlkey=12n2ml5ump0lwo434...

[John R Ellis]

[This post contains formatting and embedded images that don't appear in email. View the post in your Web browser.]

In that example panorama, LR 13.5.1's JPEG XL compression appears to retain slightly better detail than LR 6.14's JPEG compression at 800% zoom, though the difference isn't visible to my eye at 100%.

800% zoom:

[reproo2773183]

Hi John,

Your forum screengrab(s) is/are jpeg. If you open full size you can clearly see the JPEG boundaries. Just for anyone else worried about this, its just the forum-upload or screengrab settings, none of these boundaries appear in any of the RAW or DNG files John shared.

I've downloaded and crawled over your sample files, I've opened them all as 16bit RGB in Photoshop (via ACR). I can't find any jpeg artifacts (jpeg noise nor boundaries) in any of the dngs. The Noisiest most artifacty in appearance are the ARWs I'm guessing that my noise and sharpening defaults in ACR are less than optimal.

The pano6.14 is noticably softer than all the others, but I imagine if you upped the sharpening/texture/clarity by 1 or 2 it would compensate for the drop in noise that is losing detail.

I'm really struggling to see any differences between any of the other panos that aren't caused by a difference in stitching (section of the pano is coming from a different ARW or the join is in a different place). v9.1 maybe a bit more contrasty at the edges but its so subtle I might just be imagining it.

I'm going to try and find a tough image and compare but for me I'm ready to embrace JPEG XL as soon as its supported directly in Photoshop.

[Rikk Flohr: Photography]

@madbrain76 

The statement is correct. References to "Deflate" pertain to the compression of Alpha channels in the irregular boundaries created in a Panorama Merge. The RGB pixels themselves were not compressed by Deflate. 

[ACEkin]

I knew something must have changed in DNG files not because I detected quality loss but because I lost the ability to view them in external viewers. Viewing DNG panoramas in Windows File Explorer, FastStone, or XNView image viewers has become impossible. Is there a workaround to be able to browse folders and see these files? Out of curiosity, does "lossy" DNG compression still maintain the color space and 16-bit image information?

[madbrain76]

I noticed that as well.  Even after downloading Adobe's DNG converter, which includes the CODEC, Panorama and HDR DNG files don't show in Explorer. It's a little hard to believe that there is no Windows shell extension for JPEG-XL, 7 years into the start of its development.

[Uneternalism]

I just came across this change. But it also breaks the preview thumbnails in Windows compared to older DNG versions. Is there going to be any codec that allows image previews to be displayed in Windows? It is a little cumbersome to always have to open Adobe bridge to see the image previews.