Lightroom has removed 7 years of metadata from my photos - how to prevent again

Report · Nov 21, 2018

I have always added keywords, location, my address to my photos before loading them into Lightroom. I wanted to do it this way so this data always stayed with the photo and I did not have to worry about a sidecar file.

I used to use a program called Microsoft pro photo tools and it did just what I wanted. This program no longer works as the Bing API has changed and so I have been looking for an alternative program.

During the course of my test to find alternative software I found some metadata was missing from an image and looking at the file I now have a block of information like this:

<?xpacket begin='ï»¿' id='W5M0MpCehiHzreSzNTczkc9d'?><x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:Description rdf:about="" xmlns:xmp="http://ns.adobe.com/xap/1.0/"><xmp:Rating>0</xmp:Rating></rdf:Description></rdf:RDF></x:xmpmeta>

<?xpacket end='w'?>

I have done a test and Lightroom inserts this block into my CR2 file when it is imported. It seems when this block of information is inserted the original metadata gets deleted ( truncated example from an old image not imported into Lightroom 😞

<x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" xmlns:xmpRights="http://ns.adobe.com/xap/1.0/rights/"><xmpRights:WebStatement>www.mysite.com</xmpRights:WebStatement></rdf:Description><rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" xmlns:dc="http://purl.org/dc/elements/1.1/"/><rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" xmlns:photoshop="http://ns.adobe.com/photoshop/1.0/"><photoshop:AuthorsPosition>Mr</photoshop:AuthorsPosition><photoshop:Country>England</photoshop:Country><photoshop:Headline>10k Run</photoshop:Headline><photoshop:State>Cambridge</photoshop:State></rdf:Description><rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" xmlns:Iptc4xmpCore="http://iptc.org/std/Iptc4xmpCore/1.0/xmlns/"><Iptc4xmpCore:CreatorContactInfo><rdf:Description xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><Iptc4xmpCore:CiAdrCtry xmlns:Iptc4xmpCore="http://iptc.org/std/Iptc4xmpCore/1.0/xmlns/">England</Iptc4xmpCore:CiAdrCtry><Iptc4xmpCore:CiAdrRegion xmlns:Iptc4xmpCore="http://iptc.org/std/Iptc4xmpCore/1.0/xmlns/">Cambridge</Iptc4xmpCore:CiAdrRegion><Iptc4xmpCore:CiEmailWork xmlns:Iptc4xmpCore="http://iptc.org/std/Iptc4xmpCore/1.0/xmlns/">my@email.com</Iptc4xmpCore:CiEmailWork></rdf:Description>

</Iptc4xmpCore:CreatorContactInfo></rdf:Description><rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" xmlns:dc="http://purl.org/dc/elements/1.1/"><dc:creator><rdf:Seq xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:li>Anthony Barnett</rdf:li></rdf:Seq>

</dc:creator><dc:subject><rdf:Bag xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:li>10k Run runners</rdf:li></rdf:Bag>

</dc:subject><dc:title><rdf:Alt xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:li xml:lang="x-default">10k Run</rdf:li></rdf:Alt>

</dc:title><dc:description><rdf:Alt xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:li xml:lang="x-default">10k Run</rdf:li></rdf:Alt>

</dc:description></rdf:Description><rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" xmlns:MicrosoftPhoto="http://ns.microsoft.com/photo/1.0/"><MicrosoftPhoto:LastKeywordXMP><rdf:Bag xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:li>10k Run runners</rdf:li></rdf:Bag>

</MicrosoftPhoto:LastKeywordXMP></rdf:Description><rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" xmlns:MicrosoftPhoto="http://ns.microsoft.com/photo/1.0/"><MicrosoftPhoto:LastKeywordIPTC><rdf:Bag xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:li>10k Run runners</rdf:li></rdf:Bag>

</MicrosoftPhoto:LastKeywordIPTC></rdf:Description><rdf:Description rdf:about="uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b" xmlns:xmp="http://ns.adobe.com/xap/1.0/"><xmp:CreatorTool>Microsoft Pro Photo Tools</xmp:CreatorTool></rdf:Description></rdf:RDF></x:xmpmeta>

<?xpacket end='w'?>

To say I am not impressed by this is an understatement; I did this religiously every time I stored any photos onto my hard drive and I have now lost the metadata from every photo I have taken since 2011.

As I say I still have the sidecar file and I suppose I could go back over every photo and find some way to write this data back in.

My questions are:

Why is Lightroom doing this? I was always told Photoshop products never modified the original file and they never wrote to proprietary RAW files anyway.

What can I do to stop it doing it again?

If Lightroom can do this why can't it write the metadata into the file?

I know I can convert my files to .dng and then write the metadata to them but will this data be automatically written to the .dng file from the contents of the sidecar file - if the data I want is in the sidecar file? I have about 600GB of photos

I also want to know why Lightroom never warned me this would happen.

Report · Nov 21, 2018

I'd be very surprised if LR inserted that block of XMP into raw files. You were told what is everyone's understanding, that LR doesn't modify raw files. (There is one minor exception: The option Catalog Settings > Write Date Or Time Changes Into Proprietary Raw Files causes LR to overwrite the capture date.) Assuming this remains true, there must be some other explanation for your data loss.

The XMP block you included above is typically inserted by cameras that allow you to rate photos in-camera. The rating is stored in the industry standard XMP:Rating field.

Can you reproduce the behavior with a fresh photo that hasn't been previously imported into LR? Make a copy of the photo and import it to verify the problem. Then upload the original to Dropbox or similar and post the sharing link here. If you can reproduce the behavior, I think Adobe (and many users) would like to see a bug report filed.

Report · Nov 22, 2018

Thank you for the reply @johnrellis

The first example block of xmp data above was inserted into a new image when it was imported into Lightroom as you suggested.

I checked the file with a text editor before it went into Lightroom - no xmp code - and when it was imported into Lightroom - now has the xmap code.

It is quite easy for somebody else to try as you explained how and that would be helpful as it could be my PC.

I am off to work now and will post a before and after photo later.

Report · Nov 22, 2018

To clarify my previous post: Many cameras insert a similar block of XMP when they create the raw file. For example, I downloaded this raw file from DPReview.com's sample gallery: Panasonic LX100 II sample gallery: Digital Photography Review . It contains this block of XMP:

<?xpacket begin='' id='W5M0MpCehiHzreSzNTczkc9d'?><x:xmpmeta xmlns:x="adobe:ns:meta/"><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"><rdf:Description rdf:about="uuid:d874e788-25f8-4d1d-947a-6e77822b5d6a" xmlns:xmp="http://ns.adobe.com/xap/1.0/" xmlns:dc="http://purl.org/dc/elements/1.1/"><xmp:Rating>0</xmp:Rating></rdf:Description></rdf:RDF></x:xmpmeta><?xpacket end='w'?>

This is how cameras insert the XMP:Rating field into photo metadata.

I tried importing a couple of raw files that didn't contain this XMP, and I couldn't find any instances where LR 8.0 added the XMP to the raw file. So if you could post an example, that would help troubleshoot what's going on.

Report · Nov 22, 2018

I must apologise everything looks OK after all. My PC is finding it hard work checking these large files in the text editor and I must have missed something while the computer was still working.

It looks like Canon writes the empty metadata block in the original post as standard near the top of the CR2 file and the custom metadata is written towards the bottom of the file and I somehow missed it. So I actually have two lots of XMP data blocks in each file.

Report · Nov 22, 2018

No need to apologize -- metadata is a complicated mess.

The most authoritative tool for examining metadata is the free command-line utility ExifTool. To see all the metadata in a photo, you'd do:

exiftool -a -G file

There are a couple of LR plugins for invoking ExifTool, and a couple of GUI utilities layered on top of ExifTool.