P: Before after comparison of metadata conflicts

When LrC gives me a message that the photo has been changed in an external app, then it knows what has changed or it can't give me the message. So why not show me? Why not tell me what has changed? It's pretty useless to try to make a decision about whether to overwrite or not if I don't have any idea. And guessing games really aren't that much fun. If I could see the values that are different, then I could make a decision. This is probably my most frustrating thing about LrC. Please consider this going forward. Thanks.

 

Merged with this thread by Moderator

If you don't use other programs to modify photo's metadata (e.g. Bridge), then it is always safe to to overwrite the file metadata with that from the catalog. Most instances of LR's "conflict" notifications are spurious.

For me, it's because I don't know what I'm changing when I overwrite.  The fear is that I would be over-writing deliberate changes I made in the past, such as new keywords, and lose the work I did.

I filed a new bug report with a (hopefully) easily reproduced recipe for triggering spurious metadata conflicts:

https://feedback.photoshop.com/photoshop_family/topics/lightroom-scan-for-metadata-updates-causes-spurious-metadata-conflicts

The problem in the past is that Adobe hasn't been able to reproduce the problem. If they can reproduce the problem with my recipe, perhaps they'll finally fix it.

Thanks. In some recent experiments, I noticed that the number of false metadata conflicts can increase quite a bit if the network volume's clock is more than 5 seconds ahead of the computer's clock.  That can happen sometimes if the synchronization got disabled somehow or stops working.

But it doesn't look like the issue here. There are other reports on the forums about this, and many of them involve NAS. I'm going to keep poking around a bit more, but I'm not optimistic I'll find out any more.  

The server is Unraid, Unraid keeps its time in sync using network time.
From the test we can see the time is off by at most a second.

forfiles is not supported on UNC paths (I do not use mapped drives), so I did the same in powershell.

$testfile = "\\server-1\transfer\junk.txt"
Remove-Item $testfile
Get-Date
New-Item -Path $testfile -ItemType "file" -Force
Get-Item $testfile | Format-List

PS C:\Users\piete> $testfile = "\\server-1\transfer\junk.txt"                                                           PS C:\Users\piete> Remove-Item $testfile                                                                                PS C:\Users\piete> Get-Date                                                                                             
Wednesday, November 20, 2019 7:59:24 AM


PS C:\Users\piete> New-Item -Path $testfile -ItemType "file" -Force                                                     

    Directory: \\server-1\transfer


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----       11/20/2019   7:59 AM              0 junk.txt


PS C:\Users\piete> Get-Item $testfile | Format-List                                                                     

    Directory: \\server-1\transfer



Name           : junk.txt
Length         : 0
CreationTime   : 11/20/2019 7:59:23 AM
LastWriteTime  : 11/20/2019 7:59:23 AM
LastAccessTime : 11/20/2019 7:59:23 AM
Mode           : -a----
LinkType       :
Target         :
VersionInfo    : File:             \\server-1\transfer\junk.txt
                 InternalName:
                 OriginalFilename:
                 FileVersion:
                 FileDescription:
                 Product:
                 ProductVersion:
                 Debug:            False
                 Patched:          False
                 PreRelease:       False
                 PrivateBuild:     False
                 SpecialBuild:     False
                 Language:




PS C:\Users\piete>     

"my images are on a NAS (catalog on local SSD), and like most Linux based NAS's"

Which NAS device are you using?

I think there's a bug where LR gets confused when the computer's clock isn't closely synchronized with the NAS's clock.   To measure the difference between clocks in your configuration (I'm assuming you're on Windows):

1. Start the Command Prompt.

2. Copy and paste these lines to the Command Prompt:

cd /d path
del junk.txt
echo %date% %time% 
copy nul: junk.txt 
forfiles /m junk.txt /c "cmd /c echo @file @fdate @ftime"

where you've replaced path with the full path of a folder on your NAS. Copy and paste the entire output in your reply.

For example:

c:\Users\john>cd /d z:\volumes\john\desktop
z:\Volumes\john\desktop>del junk.txt
z:\Volumes\john\desktop>echo %date% %time%
Tue 11/19/2019 21:32:23.92
z:\Volumes\john\desktop>copy nul: junk.txt
        1 file(s) copied.
z:\Volumes\john\desktop>forfiles /m junk.txt /c "cmd /c echo @file @fdate @ftime"
"junk.txt" 11/19/2019 10:32:23 PM

 

I carefully compared "lr.info.PV_20130620_0509.txt" with "exiftool.info.PV_20130620_0509.txt" (I know how the fields match up), and I couldn't find any differences.  But it's entirely possible I missed something, given the manual nature of the process.

The differences before and after overwriting are expected and normal:

$ diff lr.info.PV_20130620_0509.txt lr.info.overwrite.PV_20130620_0509.txt 
56c56
<     metadataDate = "11/19/2019 5:37:56 PM", 
---
>     metadataDate = "11/19/2019 5:56:16 PM", 
114c114
<     metadataDate = "2019-11-19T17:37:56.616-08:00", 
---
>     metadataDate = "2019-11-19T17:56:16.131-08:00", 

The metadata date field is supposed to contain the date/time that an application last modified the metadata in the file. When you do Save Metadata To File, LR will update the metadata date to "now".

Similarly for the ExifTool output:

$ diff exiftool.info.PV_20130620_0509.txt exiftool.info.overwrite.PV_20130620_0509.txt 
5c5
< [File]          File Modification Date/Time     : 2019:11:19 17:38:11-08:00
---
> [File]          File Modification Date/Time     : 2019:11:19 17:56:17-08:00
164c164
< [XMP]           Metadata Date                   : 2019:11:19 17:37:56-08:00
---
> [XMP]           Metadata Date                   : 2019:11:19 17:56:16-08:00
168c168
< [XMP]           Instance ID                     : xmp.iid:f26a1af2-7143-3f4e-9392-456ef76f0c31
---
> [XMP]           Instance ID                     : xmp.iid:2ecc9d25-2119-9f46-ae6e-ac9fc3ee998e
172,173c172,173
< [XMP]           History Instance ID             : xmp.iid:f26a1af2-7143-3f4e-9392-456ef76f0c31
< [XMP]           History When                    : 2019:11:19 17:37:56-08:00
---
> [XMP]           History Instance ID             : xmp.iid:2ecc9d25-2119-9f46-ae6e-ac9fc3ee998e
> [XMP]           History When                    : 2019:11:19 17:56:16-08:00

These differences are expected when you do Save Metadata To File.

I do suspect it has something to do with timestamps, where LR may be using the file timestamp instead of metadata timestamps.

In the zip file are a few text files: https://1drv.ms/u/s!AqeqhT0NczrNk58f3keChaHYT8GXjw?e=cD8kRo
exiftool.info.[filename].txt = exiftool output.
exiftool.info.overwrite.[filename].txt = exiftool output after doing a overwrite settings.
lr.info.[filename].txt = plugin data.
lr.info.overwrite.[filename].txt = plugin data after overwrite settings.

It was not obviously easy to compare the plugin and exiftool output in a diff, but it was obvious when comparing the exiftool and plugin info before and after a save, that the timestamps changed.

The files that come back with changed information appear to be semi random, sometimes they go away after an overwrite, other times they come back.
Some mp4 files are permanently in a metadata unknown state, and do not offer a write metadata option.

I do not have any cloud syncs active. (I've tried migrating to LRCC, but it always crashes in the process, and as such I don't yet trust the result to give up on a local copy)

One other test is to turn off Cloud sync (if you're using it).  I can imaging that in some way a cloud sync operation my adjust something in metadata that triggers the mismatch.