P: Invalid XMP metadata written to exported JPEGs, tripping up reading software

10 Votes
Explorer ,
May 01, 2015 May 01, 2015

Copy link to clipboard

Copied

Lightroom 6 often writes invalid XMP metadata in exported JPEGs, including for photos containing lots of brush strokes made with the adjustment brush. This trips up Google Photos, preventing it from showing any of the EXIF metadata. It may well trip up other software.

To reproduce:

1. Start with any image.

2. Use a small adjustment brush with Exposure = 100.

3. Make lots and lots of brush strokes (see the example pic below).

4. Export the image as a JPEG, including all metadata.

5. Load the image to Google Photos and observe that it doesn't show any EXIF metadata.

6. Delete all of the XMP develop settings with:

exiftool -xmp-crs:all= file.jpg

7. Upload that modified file to Google Photos and observe that it now shows the EXIF metadata.

Here's an example pic exported from step 4:

https://dl.dropboxusercontent.com/u/2...

If you extract the XMP metadata with:

exiftool -a -m -b -xmp file.jpg

you'll see that LR has recorded all of the develop settings twice, including all the brush strokes.

Worse, if you examine the file layout with:

exiftool -m -htmldump file.jpg > file.htm

you'll see that the APP1 Extended XMP segments recording the second copy of the develop settings have incorrect segment offsets, with the first segment of the duplicate settings having offset 0, and the following segments with offsets based on that. That's not allowed by the XMP standard, which requires all the segments to have linearly increasing offsets with no gaps. It's easy to imagine how this might trip up a program trying to read the metadata.

Bug Fixed
TOPICS
macOS , Windows

Views

192

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
46 Comments
Explorer ,
May 01, 2015 May 01, 2015

Copy link to clipboard

Copied

See this thread for users encountering this problem: https://forums.adobe.com/message/7502...

Votes

Translate

Translate

Report

Report
Explorer ,
May 01, 2015 May 01, 2015

Copy link to clipboard

Copied

A workaround is to use the Metadata Wrangler plugin on your exports: http://regex.info/blog/lightroom-good...

Choose the option to remove the XMP "crs" block (which contains the develop settings).

Votes

Translate

Translate

Report

Report
Enthusiast ,
May 02, 2015 May 02, 2015

Copy link to clipboard

Copied

I'm one if the users affected by this issue in the thread referenced by John.

This is causing me a bug problem and I hope it gets fixed quickly; a DAM application generating invalid XMP metadata is unacceptable.

Votes

Translate

Translate

Report

Report
New Here ,
May 06, 2015 May 06, 2015

Copy link to clipboard

Copied

I'm having the same problem with my Phorganizer Android app. It says it can't find any EXIF data with JPGs exported with LR6. But it can find the EXIF data with JPGs exported with LR5.7 Also with JPGs exported with LR6, the Gallary Android app slideshow function doesn't display the photos in shot date order anymore. It displays the photos in file "date modified" order.

Hope Adobe fixes this soon!

Votes

Translate

Translate

Report

Report
Community Beginner ,
May 07, 2015 May 07, 2015

Copy link to clipboard

Copied

Same problem here (among many others)! Culling my photos with Photo Mechanic and having set most metadata there when I import them to LR6 metadata are invalid for jpgs.

Votes

Translate

Translate

Report

Report
Community Beginner ,
May 08, 2015 May 08, 2015

Copy link to clipboard

Copied

Came across this issue when Picasa would not see some of the EXIF metadata stored in files (photo location and exposure details among others).

Crazy thing to see such an important aspect of the software goofed up. Hoping for a point release sometime soon!

Votes

Translate

Translate

Report

Report
Explorer ,
May 11, 2015 May 11, 2015

Copy link to clipboard

Copied

Actually, this is allowed by the XMP specification, which states:

"The GUID is also stored in the StandardXMP as the value of the xmpNote:HasExtendedXMP property. This allows detection of mismatched or modified ExtendedXMP. A reader must incorporate only ExtendedXMP blocks whose GUID matches the value of xmpNote:HasExtendedXMP"

The GUID of the second extended XMP doesn't match, so technically it should be ignored. ExifTool doesn't ignore it though.

- Phil

Votes

Translate

Translate

Report

Report
Explorer ,
May 11, 2015 May 11, 2015

Copy link to clipboard

Copied

Good catch, I didn't notice the different GUIDs, thanks.

So this issue now falls into the same category as the other JPEG format issue: http://feedback.photoshop.com/photosh.... While technically conforming to the industry standard, the atypical layout of the XMP information, surely not intended by the developers, will cause LR's users significant practical problems:

1. Software like Google Photos/Picasa chokes on the XMP metadata and refuses to show any metadata at all. If Exiftool, the widely acknowledged authority on metadata, didn't properly handle LR's layout of XMP metadata, what's the likelihood of the typical mediocre software app handling it properly?

2. Duplicating the XMP-crs information twice can add a lot of bloat to files containing lots of adjustment-brush strokes. For example, it could take 1.5 MB or more to represent the strokes in a photo that's had a lot of brush work. This bug causes that to be duplicated, adding another 1.5 MB to the file size needlessly. This could be significant when publishing Web-quality photos. E.g. a high-quality 2400-wide JPEG might take 1 MB without metadata; with the XMP metadata stored properly, that would expand to 2.5 MB; with the XMP metadata stored by LR 6, it would expand a further 60% to 4 MB. Telling a user not to export any metadata isn't always an option, because she may well want EXIF camera info, keywords, captions etc. exported, and LR natively doesn't provide any mechanism for easily suppressing the develop settings only.

Votes

Translate

Translate

Report

Report
New Here ,
May 11, 2015 May 11, 2015

Copy link to clipboard

Copied

Thanks John! As you pointed out, this was "surely not intended by the developers".

Votes

Translate

Translate

Report

Report
Adobe Employee ,
May 12, 2015 May 12, 2015

Copy link to clipboard

Copied

John,

Just to be clear, if you omit step 6 in the steps of your original post, do you still see a problem in Picassa?

Thank you,
David

Votes

Translate

Translate

Report

Report
Explorer ,
May 12, 2015 May 12, 2015

Copy link to clipboard

Copied

Yes, if you don't do step 6 (remove all the XMP-crs metadata), then you can't see any of the EXIF metadata in Google Photos.

Votes

Translate

Translate

Report

Report
Adobe Employee ,
May 12, 2015 May 12, 2015

Copy link to clipboard

Copied

Thanks -- I sort of misread your original post.

Votes

Translate

Translate

Report

Report
New Here ,
May 12, 2015 May 12, 2015

Copy link to clipboard

Copied

Thanks David for looking into this. And thanks again John for your work pin-pointing this situation. Hope this gets corrected soon.

Votes

Translate

Translate

Report

Report
New Here ,
May 15, 2015 May 15, 2015

Copy link to clipboard

Copied

Same problem! Thought I was going mad! I've got a website which displays the EXIF data of uploaded images and it displays garbled up and useless information since the update to Lighroom 6! This needs to be fixed ASAP as it's a KEY feature for most Photographers to have accurate EXIF data in their images!

Votes

Translate

Translate

Report

Report
New Here ,
May 15, 2015 May 15, 2015

Copy link to clipboard

Copied

Camera Make (Manufacturer) : Empty (used to display NIKON CORPORATION)
Camera Model: CORPORATION (used to display NIKON D750)
Photographer: xx xx:xx:xx (used to display Photographer's name)
Copyright: shows Photographer's last name (used to display camera copyright text)

Votes

Translate

Translate

Report

Report
New Here ,
May 15, 2015 May 15, 2015

Copy link to clipboard

Copied

Same problem! Thought I was going mad! My website relies on the EXIF data of uploaded images and it displays garbled up and useless information since the update to Lighroom 6! This needs to be fixed ASAP as it's a KEY feature for most Photographers to have accurate EXIF data in their images!

Some examples:

Camera Make (Manufacturer) : Empty (used to display NIKON CORPORATION)
Camera Model: CORPORATION (used to display NIKON D750)
Photographer: xx xx:xx:xx (used to display Photographer's name)
Copyright: shows Photographer's last name from field above (used to display camera copyright text)

Votes

Translate

Translate

Report

Report
LEGEND ,
May 15, 2015 May 15, 2015

Copy link to clipboard

Copied

I suspect you're encountering a different problem. Does this problem occur for photos that don't have a lot of brush strokes from the adjustment brush? If so, then it's the other problem: http://feedback.photoshop.com/photosh....

Votes

Translate

Translate

Report

Report
New Here ,
May 15, 2015 May 15, 2015

Copy link to clipboard

Copied

Thanks, didn't realise there were two bugs! That other link looks more likely as there are no brush strokes used here, just standard adjustments.

Votes

Translate

Translate

Report

Report
LEGEND ,
Jun 10, 2015 Jun 10, 2015

Copy link to clipboard

Copied

A workaround is to set the export option Limit File Size To to be a very high number, e.g. 30000 K (30 MB). This was suggested by an Adobe employee in another bug report that might have the same cause as this issue:

http://feedback.photoshop.com/photosh...

I've tried a couple of exports and it seems to work.

Votes

Translate

Translate

Report

Report
New Here ,
Jun 11, 2015 Jun 11, 2015

Copy link to clipboard

Copied

First off, thank you so much John for all your hard work. I know it takes a lot of time & effort to figure out this stuff. Adobe should put you on their payroll.

So I tried some cursory tests with this Limit File Size workaround. Now the Android Gallery Slideshow function seems to work better. But still there are some photos that are out of sequence when they were taken within the same second (used to sequence down to Sec./100 I think, no time to research this in detail right now). Also, now one export process runs at 100% CPU. I used to run three parallel exports to use up CPU and still had a little CPU left. Now it's too frustrating getting LR6 to respond quickly enough so I can get the other exports started.

Unfortunately, I don't have the time to do more in-depth research and testing on this. I need to get my work out the door and don't want to redesign/change my work flow processes because of this LR 6 issue. So I back to using LR 5.7.1 even though it doesn't support my GPU & 4K monitor well. Nevertheless, it would be nice to have LR 6 working like LR 5.7.1 EXIF data wise.

Votes

Translate

Translate

Report

Report
LEGEND ,
Jun 11, 2015 Jun 11, 2015

Copy link to clipboard

Copied

"Also, now one export process runs at 100% CPU. I used to run three parallel exports to use up CPU and still had a little CPU left. Now it's too frustrating getting LR6 to respond quickly enough so I can get the other exports started."

LR 6 uses all your processors to do an export, making each export go faster, so you don't need to split a large export into smaller pieces and fire them off manually, which is what people used to do in LR 5. But I've seen a lot of reports that LR is unusable during an export now, which suggests it's being too aggressive at using all available CPU for the export.

Votes

Translate

Translate

Report

Report
New Here ,
Jun 11, 2015 Jun 11, 2015

Copy link to clipboard

Copied

Thanks John for your reply. I was unaware that LR 6 uses all my processors at 100% for exports compared to LR 5. I've only run one or two large export batches using LR 6 and didn't see the 100% CPU until my "Limit File Size" test. And I guess it would be much easier to start one large batch export anyway.

However, since realizing the EXIF data problem a month ago (which messed up my work flow processes with my customers), I haven't been using LR 6 at all. There is much less risk running LR 5.7.1 for my production work since I know it works within my work flow.

Moreover, using the "Limit File Size" workaround, any ideas about photos taken within the same second? I think the IPTC "Date Created" goes out to 1/100 sec. and maybe LR 6 is truncating the 1/100 sec. part (haven't had time to looking to this). Any help would be appreciated.

All in all, I would love to use LR 6 for its use of my GPU and increasing its response when using my 4K monitor. But right now, I can't trust it for my production work.

Anyway, Adobe should hire you. You are great. Thanks again!
Frank

Votes

Translate

Translate

Report

Report
LEGEND ,
Jun 11, 2015 Jun 11, 2015

Copy link to clipboard

Copied

"any ideas about photos taken within the same second? I think the IPTC "Date Created" goes out to 1/100 sec. and maybe LR 6 is truncating the 1/100 sec. part (haven't had time to looking to this)."

I just did a quick test, and LR 6's export does preserve fractional seconds in capture time. Note that the EXIF and XMP capture-time fields can store fractional seconds, but IPTC cannot. So when you export a photo whose capture time has fractional seconds, LR will properly set EXIF:DateTimeOriginal, EXIF:SubSecOriginal, and XMP:DateCreated to have the fractional seconds. But it will set IPTC:TimeCreated to omit the fractional seconds.

Votes

Translate

Translate

Report

Report
New Here ,
Jun 13, 2015 Jun 13, 2015

Copy link to clipboard

Copied

Thanks John for verifying this for me. However, given my cursory test, the sequencing in Android's Gallery Slideshow is off when displaying photos taken within the same second.

Never-the-less, once I get a chance, I think I have a test in mind to verify the sequencing in Android's Gallery Slideshow. I'll upgrade one of my smaller LR 5.7.1 catalogs to LR 6. Run an LR 6 export with the Limit File Size workaround. Then see if Android's Gallery Slideshow sequences the same sub-second photos the same way as photos exported with LR 5.7.1.. I hope to do this test next week, maybe Thursday or Friday. I'll get back to you with my result.

Votes

Translate

Translate

Report

Report
New Here ,
Jun 16, 2015 Jun 16, 2015

Copy link to clipboard

Copied

I just notice that the LR6.1 June update is ready for download, but I haven't tried it yet. Has anyone tried it and if so, does it fix this issue? The "Bug Fixed" list doesn't seem to reference this issue ( https://blogs.adobe.com/lightroomjour... ). However, I hopeful that it does and they just left it off the list.

Votes

Translate

Translate

Report

Report