Known Participant

Question

Is it possible to use XMP files with DNG?

Forum|Forum|6 years ago
December 6, 2019
5 replies
11730 views

I have used XMP files since the very beginning of Lightroom. It's a handy way of having an "instant backup" of changes just in case something ever happens to my Lightroom Catalog. I love the safety net that XMP files give me.

A year and a half ago (after 15 years with Canon) I switched to the Sony a7iii. The biggest downside to that is that Sony's ARW files are 47mb each. But if I use the Adobe DNG Converter, I can losslessly compress them down to just 27mb each. As I shoot thousands upon thousands of images each month, that 20mb per image savings is huge. Especially when you take into account backups.

However, this is one big thing I hate about using DNG files. And that is that every time I make a change in Lightroom (change a star rating, adjust exposure, crop, etc) that 27mb DNG file needs to be backed up. AGAIN. Because unlike my Canon raw files, with XMP sidecar files, the DNG files are written back to whenever I make any changes.

Is there any way to change Lightroom to write to a tiny XMP file every time I make a change, rather than modifying my 27mb DNG files? I would much rather backup and rebackup tiny little XMP files. It makes no sense to have to backup an entire 27mb file just because a few bytes inside of it have changed. And it costs me money (I have to pay extra to my Internet provider as I go over their imposed limits).

So please ... is there any way to use XMP files with DNG? And if not, can you please add this to the wishlist? My hard drives and my internet bills will thank you.

U

ukbrown1

Participant

I just have this basic theory, never store more inforation that you have to and cerainly do not replicate it as incosistencies can occur between two version of say photo edits, whihc one is right.

Bob Somrak

Legend

I think XMP is a "good" EXTRA backup strategy. I do not have mine set to write continuously but have a couple smart collections of "Recently Modified" and select the photos and do an XMP save once in awhile. The XMP saves your Master Edits (the hardest thing to recreate) and Keywords which are the most important 2 things to me. If the XMP would save Virtual Copy edits like it does Snapshots than is would be a way better backup but I just "Export as Original" with a standard rename of my final virtual copies (not test ones) to solve that issue. There is at least one thread on the forum where Adobe acknowledged one release of Lightroom was messing up the Backkups. That being said, I think the catalog is fairly reliable but you can't have too many backups.

M4 Pro Mac Mini. 48GB

V

vegan hillbill

Participating Frequently

Another vote from me for the facility to use sidecars with DNG files. I've just opened my main catalog and had a noodle around with a few images for a few hours. Now the backup is re-backing up the changed dngs, over 200Gb by the looks (probably more than one editting session there by the looks). Clearly the sidecars would be nowhere near 200Gb.....

And honestly, I can't see any good reason why the option would not be included in the software to allow the customer to make an informed decision which fits their backup workflow.

D

dj_paige

Legend

You need to tell Adobe, not us other Lightroom Classic users in this forum, because we can't change the software. The proper link to do that was given above.

D Fosse

Community Expert

Maybe I'm missing something here - but why do you need xmp as backup when backing up the catalog is so much more efficient? If you later need that metadata to accompany the file, it's just a ctrl+S away. If you don't, the DNG remains pristine.

The point I'm trying to make - that others here have also tried to make - is that it's very hard to argue any practical advantage to this. In any standard workflow it just complicates everything. Feature requests with no demonstrable advantage are never going to get any traction.

I have a backup regime that automatically copies all changed and new files, triggered with a single click. I removed the Lightroom catalog from this routine at some point, because the synchronizing software treats it as a single file, and so it is fully rewritten every time. That makes it vulnerable to corruption that might propagate. Instead I rely on the built-in scheduled backup routine. The point is that the DNGs aren't touched, and so they remain pristine.

D

dwterryAuthor

Known Participant

Back in LR 1.0 we had to delete our catalogs about every 3-4000 images due to performance issues. Ever since LR 2.0 came out, that has no longer been necessary. But I started the XMP practice back with v1.0 and have kept it going just in case my LR catalog ever gets corrupted. I can literally, at any time, start over with a new catalog and lose nothing (of my settings) simply by reimporting the images.

Just a few days ago a friend of mine accidentally removed 130 images from his catalog right after he finished editing and before he exported them. OOPS! I asked if he had XMP turned on ... nope. Darn. If he had, all he would have had to do was reimport the images and he'd have all of his edits back. Instead he had to re-edit all 130 images. So XMP files can save you from even small disasters.

Anyway, at this point (seeing that there doesn't sound like there would be much interest in doing what I am asking for), I might just decide to turn of the XMP writes to avoid writing back to my DNG files. I means the loss of one level of backup ... I dunno. I'll consider it.

JohanElzenga

Community Expert

A catalog backup can also save you from small disasters, if you know how. You friend should have opened a catalog backup that still contained those images, exported the images as a new catalog, and then imported that catalog into his main catalog.

-- Johan W. Elzenga

john beardsworth

Community Expert

"However, this is one big thing I hate about using DNG files. And that is that every time I make a change in Lightroom (change a star rating, adjust exposure, crop, etc) that 27mb DNG file needs to be backed up. AGAIN. Because unlike my Canon raw files, with XMP sidecar files, the DNG files are written back to whenever I make any changes. "

Obviously that's a suboptimal backup plan - both because of space demans and because XMP doesn't contain all the work you've done on the images.

Instead, back up the DNG once when it's new, and routinely back up the catalogue.

JohanElzenga

Community Expert

No, DNG files do not need XMP sidecar files, because their embedded metadata can hold all the info. And so sidecar files are not supported with DNG by design.

This is a user-to-user forum. For feature requests, go to https://feedback.photoshop.com. Search for your feature request first. If it is already requested by someone else, then don't create a new thread but add your vote to the existing request.

-- Johan W. Elzenga

D

dwterryAuthor

Known Participant

Yes, I realize they designed it that way. But now I've provided a big reason (one that is important to me, and could be to others) as to why that "design" is suboptimal.

I would much rather have the external XMP file. In all of these years (since LR 1.0 came out) having an XMP file for me has never been an inconvenience, so having one with a DNG wouldn't inconvenience me now. But I very much like the idea that a raw file is a pristine file - that you can trust that nobody ever touches it. That you can back it up and store it forever as an untouchable entity. But with DNG it gets altered every time I change the star rating or do anything else to the image ... it's a raw file that isn't pristine (the raw data is, but the file itself is not). And so it has to be backed up every time it gets touched. And I don't just mean that I feel compelled to back it up, I mean that my backup software which automatically backs up any file that changes on my system will automatically sense that the file has changed and send another 27mb over the internet to the backup server. So, for example... let's say I shoot 64GB worth of files for a wedding. It backs up 64GB worth of files after I get home and import the files into my system. It'll be a few weeks before I get to the wedding, but when I do, I'm going to rate all of the images. Now they get backed up again. I probably won't finish all of my edits in one day, so the next day as I continue editing... any remaining files that get changed will be backed up again. And again. And again, for as long as I keep making changes inside of Lightroom. See how this is suboptimal? Certainly it's not a lot of money, but it all adds up and is costing me extra for my Internet usage because it is metered. Whereas if it was using XMP files, then only tiny (what, 10K?) files change as I make my LR edits and that is all that has to be backed up, with XMP files the much larger raw files remain unchanged.

So for me ... DNG storing internally any changes is expensive. I would love to have an XMP option if it were available. Thank you, I'll take this to the feature request link you mentioned.

john beardsworth

Community Expert

The practice, not the design, is clearly suboptimal because you are repeatedly, unnecessarily backing up large amounts of data.

Of course, you're not the first to ask for sidecars for DNGs, but it's lazy thinking. Sidecars are just a workaround for file formats which are not publicly documented - not for formats like DNG, JPEG, TIF etc which are designed to carry their own XMP metadata. XMP is also designed primarily for communication between applications, not primarily as a backup, so it doesn't contain lots of Lightroom work. Unnecessary sidecars will also introduce potential conflicts reading metadata - eg should LR or other app prefer embedded or sidecar metadata? So sidecars for DNGs are unnecessary, don't include all your work, and will cause problems later.

Once you adopt DNG, you should fine tune your backup strategy, just like you would do after any other change to one's workflow. Back up DNGs once upon their creation, meaning all your image data is secured and your space usage is not bloated. Second, back up your work by backing up the catalogue. That gives you full backup coverage, and sensible use of space.

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded