Edits & Masks not saved in DNG format

Community Beginner ,
Dec 28, 2021 Dec 28, 2021

Copy link to clipboard

Copied

I recently returned from an African trip with 20,000 images imported into LR v.10.x. All had import presets, "core" keywords added, and conversion to DNG. I did some minor deletions, cropping, and luminance edits and had everything backed up on an external hard drive.

As per my usual workflow, I copied all of the images from the external drive into my home PC LR catalog. All was fine and I started my detailed editing and culling of images. LR v.11 came out so I updated and LOVED the new masking features. I up-dated to v.11.0.1 and v.11.1 when available. By then I had hundreds of hours in editing, using the new masking, and adding detailed key words.

Then, I copied the DNG images and LR catalog to a second external hard drive for off-site storage. To my shock, NONE of the v.11 image edits or new key words were in the copied DNG images. I tried creating a new catalog, exporting as a catalog, importing from the original catalog but none of the edits or key words are present in the copied DNG files. If I export the images (very, very slow process) all of the edits and key words are in both DNG files and JPEGs. 

Today Adobe said "Oops, we have a problem."

Any ideas on how to have a safe back-up until Adobe correct the issue in the next version update???

TOPICS
Windows

Views

231

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Dec 28, 2021 Dec 28, 2021

Copy link to clipboard

Copied

>Then, I copied the DNG images and LR catalog to a second external hard drive for off-site storage. To my shock, NONE of the v.11 image edits or new key words were in the copied DNG images.

 

You need to save the metadata and edits to DNG (Save Metadata to files found in Library Metadata menu) or have Auto Save to XMP enabled in Catalog Settings.

 

Creating a new catalog from the images doesn't help because the missing masks are stored in a supplementary catalog file ending '.lrcat-data' stored alongside the original catalog file.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Dec 28, 2021 Dec 28, 2021

Copy link to clipboard

Copied

Creating a new catalog from the images doesn't help because the missing masks are stored in a supplementary catalog file ending '.lrcat-data' stored alongside the original catalog file.

 

I think Adobe has figured this out too. When you create a new catalog, the lrcat-data is copied to the new catalog.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied

For many years I have turned this function off to improve performance as suggested by Adobe. The DNG format has always had all of the edit info and metadata. Is this something different with V.11?

 

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied

For many years I have turned this function off to improve performance as suggested by Adobe. The DNG format has always had all of the edit info and metadata. Is this something different with V.11?

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied


@lorendn wrote:

For many years I have turned this function off to improve performance as suggested by Adobe. The DNG format has always had all of the edit info and metadata. Is this something different with V.11?


 

I think you simply remember this wrong. If you turn off 'Automatically write metadata to files' in the preferences, and you also don't do that manually via the 'Metadata - Save metadata to files' menu, then the metadata will not be saved in the DNG files. This has always been that way and has not changed in version 11.

 

-- Johan W. Elzenga

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied

According to Adobe (at least up to LR v.10.x,) the save metadata is for JPEGs, TIFFs, and PDFs. In the past, all metadata were saved in the DNG format and there has never before been XMP sidecare files with DNGs - that has been on of the big advantages of the DNG raw format.

 

 

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied

I have always preferred keeping the proprietary Raw over DNG conversion. And that's been largely because of the separate sidecar XMP. I consider it better for several reasons.

 

Plus if I ever do later need a DNG file for some reason, I simply convert (or export) that then.

 

But: consider the other way round. You have converted to DNG, and then unexpectedly you later need the proprietary Raw instead (to use particular software, say). Now what?

 

As to saving metadata per se, I have found no usability issue by having this happen automatically with each edit. It has then helped me retrieve my very latest edits (even, subsequent to my latest Catalog backup), e.g. when needing to return to that backup due to Catalog corruption. Though this has not recurred for many years, having that up to date external XMP still offers me some comfort!

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied

Richard - Thank you. I understand the arguments for propriatary raw files and used them myself for about 4 years. Then, I discovered the DNG format and found the files smaller, faster (by far) in LR, and easier to share because all metadate were included. I have never once in 15 years of digital photography missed a propriatary raw file but understand that could happen.

 

DNGs have served me well and I have never had an issue with hundreds of back-ups on multiple hard drives and computers UNTIL LR v.11.x. Now, I am at a loss about how to do image backups short of exporting my entire catalog of 150,000 images which will take 2-3 days. The work flows I have used and taught others for years now leave me without edits, or without masking, or without history, or without keywords and metadata. Adobe has changed something and their tech people do not have a clue.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied

True, fast-load data may indeed be a reason for preferring DNG also file size, though this comparison of course depends on the particular Raw format in question. I expect a larger sensor resolution will magnify these considerations, but I don't think local adjustment metadata will follow since AFAIK masks don't conform quite to the full image resolution.

 

XMP local adjustment metadata has to date included the brushed mask info as well as the parametric sort, and this seems to still be the case. So the saved XMP does still get bigger on disk if you've added some brushed local adjustments for example, assuming this info has indeed been written out of course. To my knowledge the brand new cache structure sitting alongside the Catalog, is purely about holding and then updating the AI-analysed masking of Subject or else of Sky specifically. The same image may make use of that Subject / Sky analysis repeatedly within different local adjustments and I suppose virtual copies from the same image will also share in that. So there's an efficiency from caching that somewhere central and not as part of a particular image version's individual XMP (along with e.g. its brushed overrides to such a selection, its gradients, etc).

 

Accordingly, the questions remain about whether - and how - metadata is being written out; and, how immediately so if set to Automatic (this may reportedly get deferred in some circumstances, e.g. until you complete the use of a certain tool, or perhaps even wait until you move on to a different image). Also about whether the accompanying cache for Subject and Sky masks is getting successfully included in transfer, and then is getting successfully matched up with the right image in the destination setup.

 

And there will be a checklist to go through so far as Catalog aspects e.g. the Process Version used by a particular image.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Dec 28, 2021 Dec 28, 2021

Copy link to clipboard

Copied

By default, Lightroom Classic does not write your edits to DNG.

 

Any ideas on how to have a safe back-up until Adobe correct the issue in the next version update???

 

So, not a bug, working as designed, not going to be "fixed" — the user has to turn on an option to write edits to DNG.


Also, XMP is not a backup.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Dec 28, 2021 Dec 28, 2021

Copy link to clipboard

Copied

I recently returned from an African trip with 20,000 images imported into LR v.10.x. All had import presets, "core" keywords added, and conversion to DNG. I did some minor deletions, cropping, and luminance edits and had everything backed up on an external hard drive.

As per my usual workflow, I copied all of the images from the external drive into my home PC LR catalog. All was fine and I started my detailed editing and culling of images. LR v.11 came out so I updated and LOVED the new masking features. I up-dated to v.11.0.1 and v.11.1 when available. By then I had hundreds of hours in editing, using the new masking, and adding detailed key words.

Then, I copied the DNG images and LR catalog to a second external hard drive for off-site storage. To my shock, NONE of the v.11 image edits or new key words were in the copied DNG images. I tried creating a new catalog, exporting as a catalog, importing from the original catalog but none of the edits or key words are present in the copied DNG files. If I export the images (very, very slow process) all of the edits and key words are in both DNG files and JPEGs. 

Why do the DNG edits not show outside of LR?

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Advisor ,
Dec 28, 2021 Dec 28, 2021

Copy link to clipboard

Copied

Did you also transfer the previews folder along with the catalog file. If not, perhaps LrC is rebuilding the previews on the second machine.

Also, did you do a "Save" on the files in the first machine LrC? That would put all the metadata and processing information in the DNG files.

Kenneth Seals

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied

Yes and yes - but the metadata and edits are not in the copied DNGs as they have always been in the past.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Advisor ,
Dec 28, 2021 Dec 28, 2021

Copy link to clipboard

Copied

This is a duplicate post. Perhaps the Mods can merge them.

Kenneth Seals

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied

At lorendn, you stated,

“Why do the DNG edits not show outside of LR?”

The only application that can read those LrC edits is another Lightroom app or Adobe Camera Raw plugin for Photoshop, provided the edits have been written to the file.

 

The above is relevant when you directly open the image file with the second application.

When you use the edit in function from LrC to send a file to Photoshop and the Lightroom / Camera Raw plugin are in sync the the image will be re dered automatically in PS with Lightroom edits. While you work in PS you are using a rendered file with all Lr edits and a tiff / PSD file will be created when you save.

When you use edit in function to Send a file to a third party application e.g. Topaz Labs, a rendered image is sent to the application with or without the Lr edits dependent on the choice you make, a tiff file is saved in Lr and will get updated when the third party application edits have been saved.

 

 

Regards, Denis: iMac mid-2015, 5K 27”, GPU 2GB, Ram 24GB, HDD 3TB, macOS 11.6.5,; LrC 11.4, Lr 5.4, Ps 23.3.2-ACR 14.4,; Camera OM-D E-M1.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied

That certainly seems to be the case with LR v.11.x but it has not been the case previously. I have used external means to copy DNG files for almost 10 years and have never had this issue before v.11.

 

When I orginally imported the raw images in my travel laptop (using v.10.x) and copied these images using file manager to a portable external hard drive, all of the keywords, edits, and metadata are present. It was only when I did further edits in V.11.x that NEW metadata are missing. 

 

Clearly, Adobe has changed something.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied

At lorendn, if you are responding specifically to my post earlier today I can confirm that what has Lightroom has always function the way I have described.

Lightroom will read metadata from the file is there is xmp info available at initial import, there after Lightroom reads and writes to the Catalog file by default.

If you write metadata to the file or select the automatically write to metadata function metadata and Lightroom edits will be written to the file.

However Lightroom does not read metadata from the files, you will have to manually do the function in the Library menu.

When you are working with Lightroom if functions with the info in the Catalog and that info is written as you work.

Writing to the file is secondary and essentially is to facilitate Adobe Camera Raw to access the info since it cannot read or write to the Lightroom Catalog file.

To wit Lightroom Classic is a completely independent and fully functional application that does not require you to have any other Adobe Application installed on your computer.

I have used Lightroom from since its inception in Feb 2007 and during the beta testing from June 2006.

 

Regards, Denis: iMac mid-2015, 5K 27”, GPU 2GB, Ram 24GB, HDD 3TB, macOS 11.6.5,; LrC 11.4, Lr 5.4, Ps 23.3.2-ACR 14.4,; Camera OM-D E-M1.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied

Denis - Again, thank you for your comments. I, also, have been using LR since V.2 and have taught dozens of classes to beginner and advanced photographers. I have done hundreds of back-ups on multiple external drives and computers.

As I mentioned, the new problem with V.11.x is that information is missing from the DNG files. Every edit and keyword I added in Africa using V.10 on my travel laptop is in the DNGs and viewable from almost any application. My edits and keywords added since V.11 was installed in my desktop machine show ONLY in LR or in EXPORTS from LR and are not available in the DNG files as has always been the case.

If I open an edited DNG file now from any viewing program, only the original edits show up unless I export the file from LR. That is my confusion.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Adobe Community Professional ,
Dec 29, 2021 Dec 29, 2021

Copy link to clipboard

Copied

LATEST

Are you using Lightroom Classic on your Travel Laptop?

Two thoughts, LrC does not sync to LrC on another computer and if you are using Lr the new version I do not believe keywords sync from this Lightroom app and mobile apps. Not certain since I do not use the new cloud centric application on my computer.

 

Regards, Denis: iMac mid-2015, 5K 27”, GPU 2GB, Ram 24GB, HDD 3TB, macOS 11.6.5,; LrC 11.4, Lr 5.4, Ps 23.3.2-ACR 14.4,; Camera OM-D E-M1.

Likes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines