P: Before after comparison of metadata conflicts

Report · Dec 08, 2011

It['d be great to be able to use the before after split screen mode to look at metadata conflicts visually. You could see which version of the metadata is the one to keep. Although you still need to see differences like keywords and so on. I like rob cole's SQLLiteroom's ability to find the files with metadata conflicts, but now I need to figure out which one to keep. Even if it wasn't a visual comparison, that'd be ok. I just feel like I'm blind choosing between the catalog and the xmp file that are out of sync.

Report · Dec 10, 2011

@2011-12-10 (v4.0), ChangeManager has a feature that presents a dialog box with a list of all develop setting and metadata differences between what's in the catalog, and what's in xmp on disk. I have not thought of a way to include visual comparison, yet.

Report · Mar 01, 2017

Since Rob's website is gone, we're back to "huh, there's a metadata conflict. I have no idea what to do"

Report · Nov 09, 2017

Yes and I would like a sensible resolution choice. Right now we have:
- Overwrite original (terrible terrible terrible)
- Reload from source (looses all keywords, ratings, cropping, development, etc)

We really need a sync the metadata from file but keep Lightroom settings.

Report · Apr 13, 2018

Whenever Lightroom detects any kind of conflict between it's catalogue and a XMP sidecar file, it asks me how to handle that conflict. Problem is, I don't get any detailed information on the conflict like editing timestamps or actual differences in settings.

We do need

much better conflict detection (often times LR detects conflicts that don't exist)
overview on changes between the two versions
preview of image rendered with the different settings

This has been discussed for years now:
Lightroom: Before after comparison of metadata conflicts

Please fix this!

Report · Jun 06, 2019

I am constantly confronted with the little exclamation point icon that tells me that metadata has been changed. (Theoretically by another application, but usually no other application has been involved. Lightroom has simply lost track.)

Unfortunately, all the choices are blind.

Blindly overwrite the metadata with different metadata Lightroom has found on the disk.
Blindly overwrite the metadata on the disk with different metadata lightroom has found.
Or ignore this notice and hope it goes away.

There is no way to know what metadata has been changed. Is it EXIF data or is it develop settings? Which set of data is newest? There's no way to know.

Long ago, I learned the hard way not to click the button. I ignore it unless I absolutely need to resolve it. At that point the only way to resolve it is to create a virtual copy, update the original, and compare.

Why not add a preview feature, or better a way to compare the two versions side by side? It would save a lot of time.

Report · Jun 06, 2019

I'm glad you went to the trouble to write this one up. It doesn't sound like an easy "fix," but compared to destroying our metadata, it seems to me to be a worthwhile effort for Adobe to make. If they only let us see what we'd changed, where and when, it could be enough much of the time, but in cases like yours, maybe better to go all the way to letting us compare visually if that's reasonably possible.

Report · Jul 01, 2019

Good to know in 8 years this remains unsolved.

Obviously Lightroom can see there is a conflict, so why can't it display it? Even a basic listing of the fields it can see conflicting would be a million times better than the random guessing we have now...

In Library:
Title: "Summer in the pool"
In File:
Title "Summer in the pool!"

Report · Jul 01, 2019

Adobe just emailed an offer to "Learn from the experts on how to create amazing customer experiences." Maybe we'll get lucky and their programmers will sign up for the conference.

Report · Jul 01, 2019

Another thing I have started doing recently:

Since I automatically save metadata as external files, I have started using ‘Show in Finder’ to open the folder at the image location. There, I can make a copy of the XMP file. Then when I click the button to update the settings, I know that I can restore the original by renaming the XMP file.

Not an elegant solution by any stretch. But until Adobe decides to fix this, we’re stuck with such workarounds.

Report · Jul 01, 2019

If you're sure that you haven't modified a photo or its metadata outside of LR, it's safe to Metadata > Save Metadata To File and choose Overwrite Settings. (This doesn't address the core feature request, but LR tends to generate a lot of spurious metadata-conflict warnings.)

Report · Jul 27, 2019

I set most of the metadata (keywords, GPS position...) using Geosetter, and I develop all my photos with LR. In some occasions (like today), I forgot to reload the metadata from file before developing and then all developed photos miss the metadata.
Without the metadata comparison, I have the only choice of either lose my Geosetter settings and keep my development settings, or the opposite.
It would be nice to load metadata from file with a comparison box showing the differences between file and LR database to choose the ones to keep (for the current photo and for all photos in the batch).
Outside of LR, is there any plugin that provides this feature?

Report · Jul 29, 2019

You can download Rob Cole's ChangeManager plugin from here:
https://github.com/DaveBurns?tab=repositories

According to Rob's post from 8 years ago, it displays differences between metadata in the catalog and metadata stored in the photo or sidecar on disk. I don't know if it lets you merge the two. Many of Rob's plugins appear to work in LR 8, even though they haven't been updated in four years (Rob passed away in 2015).

Report · Aug 16, 2019

I guess that there is some information within the metadata that does not affect the appearance of the image, such as remembering how the image was printed or exported. When such entries change, there is no change in the appearance, which is the most important thing about the image.

If the above is correct, it would be useful for Lightroom to distinguish changes that affect the appearance of the image from changes that do not change that appearance. Couldn't you simply color the flag (red!?) when the differences affect the appearance, and leave it white when the appearance is not changed by the metadata difference? Using red, for example, would highlight the cases in which the user should really care about the fact that there is a conflict. And if you make such a distinction, perhaps you could add in a script that goes through a folder to remove all of those conflicts, by either updating or not (the choice set as a user preference).

Opinions???

Report · Nov 07, 2019

Out of curiosity, why don’t you click the button to resolve the conflict? Just because it’s ambiguous which is newer? Or have you also encountered issues such as file corruption?

Report · Nov 18, 2019

The metadata conflict is driving me bonkers.

I created a smart collection with metadata not up to date, about 6K out of 150K show up, select all, ctrl-s, see them saving, watch the count go down, watch the count go back up.
Select all, click conflict icon, overwrite disk, watch the count go to 0, sit back, watch the count go back up again.

LR is broken, or wrong, or both.
If we could only see what the alleged problem is...

Report · Nov 18, 2019

Do you have any plugin's active (enabled) in your LR Classic Catalog and if so which ones? I have seen cases where a plugin detects that the metadata field used to indicated a metadata mismatch has changed and in response modifies one of its own metadata fields and that change itself is then flagged as a new mismatch.

Try disabling all plugin's nut shipped with LR and see if problem persists. If it does then my speculation is incorrect. If it does correct the problem, then re-enable the plugin's one by one and test between each one to isolate which plugin is the culprit.

Report · Nov 19, 2019

I only have out of box plugins, I disabled them all, and restarted LR.
I repeated the select where status is changed on disk, save metadata, and the changed on disk count went back up as before.

I am speculating, but I wonder if LR gets confused by the file timestamps, as my images are on a NAS (catalog on local SSD), and like most Linux based NAS's it does not have nanosecond timestamp granularity. E.g. I have to use /fft when using robocopy.

If only LR would tell us what changed.

Report · Nov 19, 2019

It would be good to identify what's causing this problem. Spurious metadata conflicts have continued to be reported over the years, and while the causes of some of them have been identified and fixed, it's clear it continues to be a problem.

Here's a tedious way we can see if there's some metadata field that's causing your spurious conflicts:

1. Download and install the free ExifTool utility. (Select the Windows Executable or MacOS Package as appropriate.)

2. In LR, select one of the photos that LR shows as having a conflict.

3. Right-click the photo and do Show In Finder / Explorer to see exactly where the photo is stored on disk.

4. Start Terminal (Mac) / Command Prompt (Windows) and do this command:

exiftool -a -G "path"

where path is the full path to the file. If the photo is a raw file, change the path's extension to ".xmp". Be sure to enclose it in double quotes. For example:

exiftool -a -G "/Users/john/Pictures/2019/2019-09-30/DSC02309.xmp"

5. Copy/paste the entire output in your reply here.

6. Download my free plugin Show Catalog Metadata.

7. Unzip the download and install "showmetadata.lrplugin" into LR. If you're not familiar with how to install plugins, follow the similar instructions here.

8. In LR, with the same photo selected as in step 2, do the menu command File > Plug-in Extras > Show Catalog Metadata > Show. Copy and paste the contents of the window that appears to your reply here.

These steps will dump out nearly all of the metadata fields stored in the LR catalog with those stored in photo file on disk. We can then compare each metadata field manually to see if any are changing.

Report · Nov 19, 2019

"I am speculating, but I wonder if LR gets confused by the file timestamps, as my images are on a NAS (catalog on local SSD)"

This is definitely a possibility. I'll dig up another topic where timestamp confusion seems to be one of the causes of spurious conflicts.

Report · Nov 19, 2019

One other test is to turn off Cloud sync (if you're using it). I can imaging that in some way a cloud sync operation my adjust something in metadata that triggers the mismatch.

Report · Nov 19, 2019

I do suspect it has something to do with timestamps, where LR may be using the file timestamp instead of metadata timestamps.

In the zip file are a few text files: https://1drv.ms/u/s!AqeqhT0NczrNk58f3keChaHYT8GXjw?e=cD8kRo
exiftool.info.[filename].txt = exiftool output.
exiftool.info.overwrite.[filename].txt = exiftool output after doing a overwrite settings.
lr.info.[filename].txt = plugin data.
lr.info.overwrite.[filename].txt = plugin data after overwrite settings.

It was not obviously easy to compare the plugin and exiftool output in a diff, but it was obvious when comparing the exiftool and plugin info before and after a save, that the timestamps changed.

The files that come back with changed information appear to be semi random, sometimes they go away after an overwrite, other times they come back.
Some mp4 files are permanently in a metadata unknown state, and do not offer a write metadata option.

I do not have any cloud syncs active. (I've tried migrating to LRCC, but it always crashes in the process, and as such I don't yet trust the result to give up on a local copy)

Report · Nov 19, 2019

I carefully compared "lr.info.PV_20130620_0509.txt" with "exiftool.info.PV_20130620_0509.txt" (I know how the fields match up), and I couldn't find any differences. But it's entirely possible I missed something, given the manual nature of the process.

The differences before and after overwriting are expected and normal:

$ diff lr.info.PV_20130620_0509.txt lr.info.overwrite.PV_20130620_0509.txt 
56c56
<     metadataDate = "11/19/2019 5:37:56 PM", 
---
>     metadataDate = "11/19/2019 5:56:16 PM", 
114c114
<     metadataDate = "2019-11-19T17:37:56.616-08:00", 
---
>     metadataDate = "2019-11-19T17:56:16.131-08:00",

The metadata date field is supposed to contain the date/time that an application last modified the metadata in the file. When you do Save Metadata To File, LR will update the metadata date to "now".

Similarly for the ExifTool output:

$ diff exiftool.info.PV_20130620_0509.txt exiftool.info.overwrite.PV_20130620_0509.txt 
5c5
< [File]          File Modification Date/Time     : 2019:11:19 17:38:11-08:00
---
> [File]          File Modification Date/Time     : 2019:11:19 17:56:17-08:00
164c164
< [XMP]           Metadata Date                   : 2019:11:19 17:37:56-08:00
---
> [XMP]           Metadata Date                   : 2019:11:19 17:56:16-08:00
168c168
< [XMP]           Instance ID                     : xmp.iid:f26a1af2-7143-3f4e-9392-456ef76f0c31
---
> [XMP]           Instance ID                     : xmp.iid:2ecc9d25-2119-9f46-ae6e-ac9fc3ee998e
172,173c172,173
< [XMP]           History Instance ID             : xmp.iid:f26a1af2-7143-3f4e-9392-456ef76f0c31
< [XMP]           History When                    : 2019:11:19 17:37:56-08:00
---
> [XMP]           History Instance ID             : xmp.iid:2ecc9d25-2119-9f46-ae6e-ac9fc3ee998e
> [XMP]           History When                    : 2019:11:19 17:56:16-08:00

These differences are expected when you do Save Metadata To File.

Report · Nov 19, 2019

"my images are on a NAS (catalog on local SSD), and like most Linux based NAS's"

Which NAS device are you using?

I think there's a bug where LR gets confused when the computer's clock isn't closely synchronized with the NAS's clock. To measure the difference between clocks in your configuration (I'm assuming you're on Windows):

1. Start the Command Prompt.

2. Copy and paste these lines to the Command Prompt:

cd /d path
del junk.txt
echo %date% %time% 
copy nul: junk.txt 
forfiles /m junk.txt /c "cmd /c echo @file @fdate @ftime"

where you've replaced path with the full path of a folder on your NAS. Copy and paste the entire output in your reply.

For example:

c:\Users\john>cd /d z:\volumes\john\desktop
z:\Volumes\john\desktop>del junk.txt
z:\Volumes\john\desktop>echo %date% %time%
Tue 11/19/2019 21:32:23.92
z:\Volumes\john\desktop>copy nul: junk.txt
        1 file(s) copied.
z:\Volumes\john\desktop>forfiles /m junk.txt /c "cmd /c echo @file @fdate @ftime"
"junk.txt" 11/19/2019 10:32:23 PM

Report · Nov 20, 2019

The server is Unraid, Unraid keeps its time in sync using network time.
From the test we can see the time is off by at most a second.

forfiles is not supported on UNC paths (I do not use mapped drives), so I did the same in powershell.

$testfile = "\\server-1\transfer\junk.txt"
Remove-Item $testfile
Get-Date
New-Item -Path $testfile -ItemType "file" -Force
Get-Item $testfile | Format-List

PS C:\Users\piete> $testfile = "\\server-1\transfer\junk.txt"                                                           PS C:\Users\piete> Remove-Item $testfile                                                                                PS C:\Users\piete> Get-Date                                                                                             
Wednesday, November 20, 2019 7:59:24 AM


PS C:\Users\piete> New-Item -Path $testfile -ItemType "file" -Force                                                     

    Directory: \\server-1\transfer


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----       11/20/2019   7:59 AM              0 junk.txt


PS C:\Users\piete> Get-Item $testfile | Format-List                                                                     

    Directory: \\server-1\transfer



Name           : junk.txt
Length         : 0
CreationTime   : 11/20/2019 7:59:23 AM
LastWriteTime  : 11/20/2019 7:59:23 AM
LastAccessTime : 11/20/2019 7:59:23 AM
Mode           : -a----
LinkType       :
Target         :
VersionInfo    : File:             \\server-1\transfer\junk.txt
                 InternalName:
                 OriginalFilename:
                 FileVersion:
                 FileDescription:
                 Product:
                 ProductVersion:
                 Debug:            False
                 Patched:          False
                 PreRelease:       False
                 PrivateBuild:     False
                 SpecialBuild:     False
                 Language:




PS C:\Users\piete>

Report · Nov 20, 2019

Thanks. In some recent experiments, I noticed that the number of false metadata conflicts can increase quite a bit if the network volume's clock is more than 5 seconds ahead of the computer's clock. That can happen sometimes if the synchronization got disabled somehow or stops working.

But it doesn't look like the issue here. There are other reports on the forums about this, and many of them involve NAS. I'm going to keep poking around a bit more, but I'm not optimistic I'll find out any more.