P: Invalid XMP metadata written to exported JPEGs, tripping up reading software

[JohnRolfeEllis]

Lightroom 6 often writes invalid XMP metadata in exported JPEGs, including for photos containing lots of brush strokes made with the adjustment brush. This trips up Google Photos, preventing it from showing any of the EXIF metadata. It may well trip up other software.

To reproduce:

1. Start with any image.

2. Use a small adjustment brush with Exposure = 100.

3. Make lots and lots of brush strokes (see the example pic below).

4. Export the image as a JPEG, including all metadata.

5. Load the image to Google Photos and observe that it doesn't show any EXIF metadata.

6. Delete all of the XMP develop settings with:

exiftool -xmp-crs:all= file.jpg

7. Upload that modified file to Google Photos and observe that it now shows the EXIF metadata.

Here's an example pic exported from step 4:

https://dl.dropboxusercontent.com/u/2...

If you extract the XMP metadata with:

exiftool -a -m -b -xmp file.jpg

you'll see that LR has recorded all of the develop settings twice, including all the brush strokes.

Worse, if you examine the file layout with:

exiftool -m -htmldump file.jpg > file.htm

you'll see that the APP1 Extended XMP segments recording the second copy of the develop settings have incorrect segment offsets, with the first segment of the duplicate settings having offset 0, and the following segments with offsets based on that. That's not allowed by the XMP standard, which requires all the segments to have linearly increasing offsets with no gaps. It's easy to imagine how this might trip up a program trying to read the metadata.

[johnrellis]

The bug still exists in LR CC 2015.1. I haven't tested it in LR 6.1, but it's safe to assume it isn't fixed there either.

[SentimentalPhotos]

Thank you so much John for all your help and hard work.

But instead of me spending a lot of time trying to find the exact cause for my Android Gallery Slideshow app sequencing issue and going crazy doing it, I found a better Android app to do my customer slideshows. So I changed my workflow to use the Android QuickPic Gallery app. (I really didn't want to change my workflow, but all-in-all this change is the best option for my situation.) This app allows me to easily select, in a number of ways within a folder, a slideshow's sequencing order and the by "file name" is the selection I'm using. (You would think that the Android Gallery app would have a by file name order option, but it doesn't.). And since my file names are already named by yyyy-mm-dd-hh-mm-ss and trailing sub-second suffix if needed for uniqueness, I'm "home free" and in control of the slideshow's order no matter when this LR6 XMP issue is fixed.

Never-the-less, hope Adobe fixes this LR6 XMP issue for you soon. Good Luck!

[johnrellis]

Well, I'm glad you have a workaround.

[dfranzen_camera_raw]

John,

Can you try again to reproduce your original issue: no Exif appearing in Google Photos? I tried to today with your "example from step 4" photo today and it displays the basic Exif information. Perhaps Google Photos has fixed support for extended XMP on their end.

[johnrellis]

Agreed, it appears that Google Photos does correctly show the EXIF fields for that example photo (and a couple of new ones created with the same recipe).

However, the most recent desktop Picasa for Mac (3.9.139.218) and for Windows (3.9.139.218) still trip up on them. (I don't think Google is very actively maintaining Picasa?)

I don't have access to Android devices and thus can't test out whether they've fixed the issue in Google's Android gallery (which I think some people were complaining about).

Note that LR CC 2015.1 is still writing the XMP-crs info twice into the file, adding an extra megabyte or two for extensive brush strokes.

Thanks.

[SentimentalPhotos]

Just to report back, I have both Google's Android Gallery 5.0.1 "Lollipop" (the most up-to-date version) & 4.4.2 "Kitkat" (the previous version). Both still have the problem I described a month ago with its slideshow function. Also, many other Android photo apps (e.g., Phorganizer) I think use the same "parts" (data structure, shared code, ... ) that Gallery uses to read the .jpg file meta data so they will and do error out too. And I guess these Android apps could eventually update their code to conform with LR 6.x's .jpg meta data.

But more importantly, does Adobe considers this to be a problem with LR 6.x that needs to be fixed and will fix it? Or are we just out of luck and just need to live with it & find workarounds? My hope is that Adobe will implement a fix soon (maybe a check box within export to write the meta data like LR 5.7.1.)

Frank

[dfranzen_camera_raw]

John,

I have not been able to reproduce the problem of the XMP repeated twice. With the step 4 JPEG I do see that the JPEG has several extended JPEG blocks. It might help me if you could post a raw, DNG or JPEG file from step 3 in your original steps, but without those adjustments applied to the image yet and "baked into" the pixels. Then I could try exporting that file to see if the export produces a JPEG with multiple copies if the XMP. So far I've not seen that happen, but if you have a file that, when exported, consistently produces the result when doing an export, that would help.

Thanks,
David

[dfranzen_camera_raw]

John -- nevemind. I've found a way to reproduce the problem. Thanks.

[johnrellis]

OK, good.

[dfranzen_camera_raw]

One workaround for now would be to go back to your original file and re-export a new JPEG. That new JPEG might have extended XMP (based on the amount of metadata and the metadata export options), but it should not have duplicate copies. After export, do not make any additional metadata edits to the exported file, if it contains extended XMP.

[FeedbackCommunityMember]

Same here. Problems with Exif data in various forums and websites, including my own Wordpress site with Exif plugin installed (tried all available through WP repository). Simply seems the 'industry standard' is not so standard if every plugin under the sun used online does not conform to it. Back to old version, though I frankly find it unacceptable considering we're now essentially paying for something that fails to deliver a very basic functionality for photographers on the premiss that Adobe does it right, and everyone else should readjust. If a third party LR plugin can solve, or at least circumvent the problem, why is it such a big deal to push a hotfix addressing the issue?

[Dawid_Ciecierski]

"though I frankly find it unacceptable considering we're now essentially paying for something that fails to deliver a very basic functionality for photographers on the premiss that Adobe does it right, and everyone else should readjust"

Could not agree more. I see this plus silence on the issue (i.e. "yes this is a bug we're going to fix it in the next point release" or "sorry we are not fixing this stick to LR5") as both disrespectful of users.

[SentimentalPhotos]

I totally agree. The silence from Adobe is "deafening".

[ssprengel]

On July 7 an Adobe employee said he could reproduce the problem.

That is as much feedback as you'll probably get until the next release.

It is not disrespectful to us customers if we aren't privy to Adobe's internal project management and bug tracking systems.

[Dawid_Ciecierski]

This bug is not a minor inconvenience but something that trips up pretty much everything outside of LR6, including our clients' software. If you're a photographer with little technical skill or not willing to fiddle with EXIFtools this is a show stopper.

All I was saying was that I see this issue as serious enough to warrant at least an acknowledgement ("yes it's a bug not feature") and confirmation this is worked on... or a "no sorry it's not getting fixed go elsewhere".

All we got in the past *three months* was "oh yes, the software does behave as you guys say it does". I expect more from a tool marketed as one for professionals who depend on it in their job.

[ssprengel]

The Adobe employee's exact phrase, above, is:

"I've found a way to reproduce the problem."

[FeedbackCommunityMember]

If they can reproduce the problem, then they're also working to solve the problem.

[SentimentalPhotos]

Thank you Chris for your official acknowledgement. Hopefully we'll see this problem fixed in LR6.2.

[johnrellis]

This problem appears to be fixed in LR 6.1.1 / CC 2015.1.1.

[SentimentalPhotos]

Great, thanks John! I'll try it out once I get a chance.

[SentimentalPhotos]

Did a quick test. Looks like it's fixed. Thanks