Exporting to DNG from Lightroom classic

[OzPhotoMan]

Hi all,

Some time ago I gave up on LRC as I did not want to be tied in to a proprietary catalog, preferring to use named folders for various versions of my files.

I have about 1000 images processed in LRC, and now wish to convert them to DNG and ditch LR altogether.

If I then re-edit those images, do I lose any quality?

Regards,

Steve

[Rob_Cullen]

Converting a proprietary raw file to the DNG format does not lose quality at all- the camera sensor data is identical in both.  (Converting RGB files- JPG, TIF, etc, is a waste of effort as the DNG only becomes a 'wrapper' for the RGB file contained within).

Exporting to DNG from Lightroom-Classic will however encapsulate LrC editing in the metadata of the DNG file. The editing metadata will only be visible in other Adobe apps that can read this metadata  (LrC, Lr, ACR, Bridge). In other software, or file viewers, the DNG will appear as original from camera.

LrC has no problem using named folders for various versions of files. You do not have to stick with the often recommended 'Dated folder' structure. Your folder structure is entirely your decison.

[D Fosse]

As Rob says. Nothing lost, all the camera sensor data are there.

But you need to do this first from Lightroom Classic:

LrC stores all edits centrally in the catalog database, not in the file. So ACR can't read it unless you first export all this metadata to the files (as a sidecar for proprietary formats, into the file header for DNG).

Just so it's said, you can use LrC entirely based on your physical folder structure on disk. That's what I do. The "Folders" panel in LrC operates just like a normal file browser. You don't need to use the catalog-based modes of organization.

[OzPhotoMan]

Interesting.  I am not sure how I have LRC set up, but if I  export one of my LRC edited file to dng, all edits are present in the dng. So it must be set up to save edits in the metada in the dng file. I have used my folder structure when I was using LRC.

My main concern was if I exported to dng from LRC with edits embedded,  if I then proceeded to do further edits would I lose any quality? It appears not.

My current PS workflow is basically to use adjustment layers on a single duplicate smart object layer for each process, such as curves, colour balance, etc so if I reopen the file later it is easy to tweak later if I want to re-edit. I also do a lot of edits in ACR first, and of course those are retained in sidecar files as well as the file itself.

[jane-e]

@OzPhotoMan 

DNGs do not use a sidecar — the edits are stored in the file. 

If you look at the images in Adobe Bridge, you will see a small icon showing that edits have been made. You can right-click the image to copy or clear the edits.

Edits in Photoshop have no effect on the RAW file.

Jane

[OzPhotoMan]

---

@jane-e wrote:

@OzPhotoMan 

DNGs do not use a sidecar — the edits are stored in the file. 

If you look at the images in Adobe Bridge, you will see a small icon showing that edits have been made. You can right-click the image to copy or clear the edits.

Edits in Photoshop have no effect on the RAW file.

Jane

---

I have set LRC to automatically write edits to xmp files. I probably did that when I installed it many years ago. If that raw image file is then exported as a dng, then when the dng is opened in ACR, then those edits are visible in ACR. 

I do realise that PS edits are not reflected in ACR or the raw file, which is why I do a lot of edits in ACR then open in PS for tweaks that cannot be done in ACR (not many in fact)

I did not know about the right click clear edit. That is good to know.

Regards,

Steve

[D Fosse]

if I  export one of my LRC edited file to dng, all edits are present in the dng. So it must be set up to save edits in the metada in the dng file.

By @OzPhotoMan

Yes, if you Export a raw file to DNG, the edit metadata will be read from the LrC catalog and included in the DNG.

Or you can save metadata to files and batch convert to DNG in Lightroom.

If you're going to convert proprietary raw files (nef/arw/cr3 etc) to DNG anyway, either method will get you there and be equally effective.

If the conversion is not necessary, you can just save metadata and the files will be ready for use in ACR, as they are.

And to be perfectly clear, all these formats are just wrappers. The camera sensor data inside that wrapper will be the same (excluding some types of optional lossy compression, which I would not recommend)

[Per Berntsen]

As far as I know, exporting as DNG will always include the edits, even when Automatically write changes into XMP is not enabled in the catalog settings. If you open an exported DNG in ACR, the edits will be applied, and all the sliders will be at the same position as they were in LrC.

As others have mentioned, you can manage your photos based on existing folders.

I use named folders, and rarely use keywords and collections, but I do use ratings and flags.

Note that the folder tree in LrC doesn't mirror the folder tree as seen in the OS.

For a folder to appear in the LrC folder panel, you need to import at least one image from it.

And the folder could contain images that don't appear in LrC because they have not been imported.

[OzPhotoMan]

As far as I know, exporting as DNG will always include the edits, even when Automatically write changes into XMP is not enabled in the catalog settings. If you open an exported DNG in ACR, the edits will be applied, and all the sliders will be at the same position as they were in LrC.

By @Per Berntsen

 Just tested this, and if write to xmp is unchecked, no xmp file is created.

As others have mentioned, you can manage your photos based on existing folders.

I use named folders, and rarely use keywords and collections, but I do use ratings and flags.

Note that the folder tree in LrC doesn't mirror the folder tree as seen in the OS.

For a folder to appear in the LrC folder panel, you need to import at least one image from it.

By @Per Berntsen

 Thanks, I do know that and in fact I work much the same. A folder for raw, one for psd and one for jpg. These are all subfolders under master folders named for categories and events by date shot, such as sport, animals, etc. Bearing in mind this is my PS workflow, but it also works for LR once the folders are imported.

Regards,

Steve

Regards,

Steve

[Rob_Cullen]

With all the great comments from others above and Reading your first post again and considering-  "..ditch LR altogether."

Then your options are-

1) In LrC:  Write metadata to proprietary raw files. This will create sidecar XMP files with the proprietary raw files. All raws will open in Photoshop (from Bridge or a file browser) through ACR with the last edit state visible. You always keep the raws with a sidecar.

2) In LrC:  EXPORT files as DNG. No sidecar is created, but the last edit state is visible from the metadata saved within the DNG file.  Now you have two files to contend with- the proprietary raw and the DNG raw. ie. more things to worry about. Are you going to delete the proprietary raw files and just keep the DNGs? Or are you going to Convert in LrC (Not Export)?

3) Keep Lightroom-Classic running and indexing all your files, then you need do nothing more. You get to keep all History, Virtual Copies, Collections, etc. And you simply [Edit in Photoshop] when you want -creating a new TIF or PSD file in the LrC catalog.

I question why you want to Export to DNG. I recommend you simply stay with Lightroom-Classic  and all these questions and hassles go away.  Lightroom-Classic is ACR !

[Thomas Dearie]

Thanks, Rob

I'm retiring Lightroom Classic and moving to DXO Photolab.
So I'm interested in quickly batch exporting the photos that have been edited so that they're editable in other photo apps (But not generate new files for unedited shots).
I have tens of thousands of images in Lightroom, so I wonder if you have a quick method to do this?

Many thanks

Tom

[ExUSA]

None of your edits will be usable in DXO software. You'll need to export to PSD or TIFF (to maintain lossless quality) and then do further editing in another tool.

[D Fosse]

And even that is far from lossless. Rendering a raw file into an RGB encoded file reduces the available dynamic range from around 14 stops down to 8 or 9. Those 5 or 6 stops are permanently hard clipped and lost.

Raw converters are all different conversion engines using different algorithms, and settings are not transferable. If you move to DXO you'll need to start entirely from scratch. As you would going the other way.

[Thomas Dearie]

Thanks folks. As expected. 

I was hoping (not optimistic) there was a meta-data translation engine available that would read xmp data and make a  translation from one platform algorithm to the other via a universal platform-independent notation, just as we've done with ICC profiles. A similar Rosetta-stone principle could be used for RAW edits.

All platforms' edits make changes to fundamental light values that aren't proprietary. So those changes could be translated from the source platform's proprietary notation to universal values, then translated to the target platform's proprietary notation.

As far as I can see, no platform has done that.

Still, the migration has to be done.
I'll have to sort and export.
Big job with thousands of files.

Cheers and thanks for the input.

T.

[D Fosse]

All platforms' edits make changes to fundamental light values that aren't proprietary. 

By @Thomas Dearie

You're missing the point. Nothing is ever changed in the file. The edits are just text instructions internal to that proprietary software. You can't get to it from the outside - it's not like the numbers in an RGB file which are out in the open and can be referenced.

[Thomas Dearie]

=: )
Thanks, but no.

The concept of preserved RAW files paired with parametric instructions is the point.
It's the instructions that require translation to move from platform to platform while retaining the RAW files intact.

All of those instructions ultimately refer to fundamental light values that are platform independent, regardless of the proprietary notation  — they have to, since every platform is dealing with the same underlying color physics.

So it's possible to wrtie an engine that translates from one proprietary notation the other.
It's a software problem, not a colour physics problem. 

It just hasn't been done.
And it may never be done, as developers may not see the advantage to themselves.
But it would be useful for professional photographers.

[ExUSA]

its not that simple. Each application implements those "light values" in different ways. Adjustment values only make sense for that vendor, and can't just be translated. Those values go into an algorithm to get to a final adjustment. Unless you propose to reverse-engineer every editing tool in every app, its not possible. Plus one app will have tools that another app doesn't.

This sounds good on paper but just can't be done.

[Thomas Dearie]

I didn't say it's simple.
That's a different discussion, and a relativistic one.

It's incorrect to say platform specific algorithms just can't be translated — all algorithms can be translated.
Algorithms are defined sets of instructions  - there's no magic to it.
In this case, those instructions are related to platform-independent light values in a fixed and defined way. that's machine-readable.

So, yes. You're reverse engineering. But that's how all software translation works.
There's nothing special about software for photo edits.

Whether software companies care to do it is a different thing. 

[ExUSA]

You obviously don't understand how any of this works. You can't translate Lightroom RAW edits so that they are usable in Microsoft Paint, for example. You have to bake them in to a rasterized file. The whole point of parametric editing data is that you can go back and change it later. If a different app has different features, its not going to be editable by definition.

And no, all algorithms can't be translated. Cryptographic functions come to mind right off. 

[Thomas Dearie]

Easy, friend,

No need to be nasty. 

There's always more to know, but I do write and teach curricula for this at a college level. So maybe I have a bit of an idea of "how any this works".   : )

Translation would be of the parametric instructions from proprietary notation of source platform to the notation of the target platform. Not the RAW data.

Writing to a non-editable format would not be needed nor any change to the base RAW data. All that's happened is that the parametric instructions have been changed from the proprietary notation of the source app to that of the target app. They're still modifying the same universal light parameters.

It's true that, if some functions are missing entirely in one app or the other, they'll be left behind.  
So yes, short of adding major functions to MS Paint, you wouldn't translate from Lr to that app.

Which I think we can all take as obvious.
And some other functions would be more challenging or complicated to port than the Basic panel — masking, particularly AI masking, for example.

But the essence of translation remains the same as it does for all software.

[ExUSA]

Nobody is being nasty, we are just trying to explain to you that what you want isn't possible. Period. That simple.

[Conrad_C]

All of those instructions ultimately refer to fundamental light values that are platform independent, regardless of the proprietary notation  — they have to, since every platform is dealing with the same underlying color physics.

By @Thomas Dearie

That might over-simplify what is actually going on. It sounds like you’re assuming all raw apps work like Curves in Photoshop: The image has an input value, then you set an output value, and to achieve the output value, the input value goes through a simple translation or algorithm.

However, many features in current photo editors are far more sophisticated than that. For example, when Adobe switched their raw processors to Process Version 2012, it was no longer just a simple formula, it became about advanced tone mapping. For example, the Highlights option incorporates some highlight reconstruction code for values near clipping (it can reconstruct by finding data in adjacent channels), and the Shadows option includes some automatic masking to prevent unwanted side-effects in non-shadow areas. They also incorporated Local Laplacian Filters. Many current features are that complex.

The point is that when you see an option labeled “Highlights” or “Shadows” in other software, it probably doesn’t work the same way, and not only that, the other software might not have the code to reproduce a result possible in other photo editors using specialized techniques. So it’s not as simple as “oh, we’ll just use the same algorithm.” (If the look of one photo editor was achieved by applying a color LUT, and the other photo editor hasn’t added support for LUTs, it may not be possible to reproduce that look exactly.) Light physics might be light physics, but you’re forgetting that digital photos are not light, they’re a representation of it, and each photo editor has evolved its own representation that is not 100% compatible with others. An analogy: We all live similar lives on the same planet, yet some languages evolved to have words that don’t exist in other languages and can’t be translated literally.

Now, there have been attempts to do what you say, sort of. If you haven’t already looked into Avalanche (commercial software), it claims to be able to transfer edits between parametric photo processors including Lightroom Classic. But they made a business decision to not take the approach you suggest, they do not attempt to translate the processing instructions. Their app (if I read their website right) uses AI to see what your edited image looks like in the source app, and then it tries to reproduce that look as closely as possible using the destination app’s options. I have never used Avalanche so I have no opinion of it, but it might be an option for you to try if it supports the applications you want to leave and to migrate to.

But even Avalanche acknowledges the challenges we are all talking about:

(*) Not all edits can be converted due to the complexity of the proprietary formats of the catalogs. Avalanche will not convert mask based edits, or local adjustments (spot removal, healing brushes, etc…). Advanced color grading is not handled either. All images that have adjustments that are not supported, will be placed in a dedicated album for easy review.

[Thomas Dearie]

Thanks, Conrad.

I'm not making those assumptions, but yes - some functions would be much more challenging than the Basic panel.

AI masking would be particularly awkward., as I noted this in an earlier post.

So no surprise at your quote from Avalanche 👍

And I make no comment as to whether software developers are motivated to go to the trouble.
I can't imagine Adobe has any motivation to have their RAW files ported to DXO. 
The same might not be true for DXO   : )

But digital photos are exactly light.
Ultimately, regardless of the approach, all apps are dealing with precisely the same underlying immutable parameters. It can't work any other way.
Software may get more complex as time goes on but, as the saying goes, if someone can write it someon can read it.

Cheers

Tom

[Conrad_C]

I'm not making those assumptions, but yes - some functions would be much more challenging than the Basic panel.

By @Thomas Dearie

Actually, that was part of my point…I was describing the the Basic panel 🙂 (Highlights and Shadows), and even those options can be challenging to understand! More recent options such as Dehaze and Reflection Removal are based on models that might be developed/trained in-house, differering with each software developer.

Furthering the point, the Highlights and Shadows options in the Basic panel are far more complex than the Highlights and Shadows options in the Parametric Curve panel. Which means, even within Lightroom Classic itself, there are options with the same name but radically different underlying code.