Skip to main content
S
Samoreen
Inspiring
July 29, 2015

P: Wrong timestamp stored in catalog causing wrong metadata status (all Windows versions)

  • July 29, 2015
  • 66 replies
  • 3424 views

Hi,

The problem I describe below is not new (I observed this at least since version 3) but this time I took the time to investigate more deeply...

From time to time, LR tells me that the XMP file of a given image is no longer in sync with the metadata in the catalog. Most often, this is correct because I made changes without recording the XMP file (Ctrl-S - I'm not using the automatic XMP updating mode). But very often, this information is simply wrong. I didn't change anything to the image and suddenly, the little down arrow appears in the upper right corner of the thumbnail.

Hitting Ctrl-S may or may not fix the problem. Sometimes, the little down arrow reappears after a few seconds or minutes although I didn't do anything (hands away from keyboard and mouse).

I recently did the following test for multiple images unduly displaying the "metadata status changed" flag. I compared the following values :

1. Windows "last modified" timestamp for the XMP file.
2. Value of the xmp:MetadataDate field in the XMP file.
3. touchTime column value for that image in the Adobe_images table of the catalog (which is a SQLite database).

The touchTime value is stored in a special format, so I had a hard time converting it to a readable date/time value. However, I will not explain this and how I navigated the database in order to access this timestamp (this requires some knowledge about databases).

Result:
For all the images tested, values #1 and #2 were always strictly identical. The touchTime value was always off (sometimes about 10-15seconds, sometimes much more). So no wonder that LR thought that the XMP file and the metadata in the catalog were not in sync.

Moreover, the difference in time can be negative of positive. So LR displays the up or down arrow accordingly (meaning that the XMP file is older or newer than the metadata in the catalog, respectively - which is wrong in both cases).

I explained above that sometimes, the image reappears as "not in sync" just a few seconds or minutes after I did a Ctrl-S. In that case, a quick look at the database showed me that the touchTime field had not been updated. So the time difference causing the image to be flagged as "not in sync" was still there. In that case, the problem can be fixed by reading the metadata from the XMP file which was actually correctly updated. This operation updates the catalog and everything is in sync again.

Anyway, there's something very wrong in the computation of the touchTime value of the Adobe_images table. That seems to be obvious. This wrong timestamp generates in turn a wrong metadata status.

I made another interesting test :

1. I started from a situation were 0 image was flagged as "not in sync" with the XMP file.

2. I purged all 1:1 previews and started a Build all 1:1 previews.

3. In Library mode, I setup a filter to show only the images that had the Metadata Status set to "has been changed". I got a cup of coffee and waited.

At the beginning of the generation process, no image was displayed in the grid, as expected. While LR was building the previews, images unduly tagged as "not in sync" started to appear. I got about 200 of them. For all these images, the metadata status was just plain wrong. These were finalized images not modified since a long time and for which the XMP file had been timely updated after the last modification. I checked the touchTime field for some of them and each time it was different from the Windows "last modified" timestamp and from the xmp:MetadataDate field of the XMP file as mentioned above.

So now I know what's going wrong but I'd like to have this problem fixed after all these years.

Thanks in advance.

    This topic has been closed for replies.

    66 replies

    Ian Lyons
    Ian Lyons
    Community Expert
    November 28, 2022

    This issue has been around since the very first builds for Windows in June 2006 (i.e. Public Beta 4). It was identified then by the Engineering Lead and multiple times by other Adobe staff ever since as a limiation of Windows. The same issue occurs with History and other areas with lists.

    M
    mlimke
    New Participant
    November 27, 2022

    I just had some success with the "failure to make metadata changes stick" problem and thought I'd share. This is when you see "unsaved changes," select the photos, save metadata, and then they quickly become unsaved again on their own.

     

    I am on Windows 10, using the Creative Cloud licensing, so I'm up to date. Most of my catalog was created with an older version of Lightroom, a standalone app from 10 years ago, I'd guess.

     

    Today, I found that this sequence helped:

    1. Select the problem photos
    2. From the menus, click Photo > Develop Settings > Reset (this is also on the right-click menu)
    3. Save the metadata to disk with Cmd/Ctrl-S

     

    The change have stayed put and now I have no more unsaved changes

     

    This did away with photo modifications such as white balance and cropping, of course, but for me, that was an acceptable tradeoff. I did fiddle with the related menu action "Update to Current Process Version," but it wasn't consistently workable. Your mileage may vary.

    S
    SamoreenAuthor
    Inspiring
    July 9, 2021

    *Bjarne Christensen 

    Hi Bjarne,

    I have given up any hope about this issue. One may wonder why such a bug has not been fixed after all these years. Laziness ? Maybe. But I think there's a more basic reason. Don't forget that a good part of LR is written in LUA, a language that doesn't make debugging and maintenance very easy. The problem with such languages is well known. Once the original developers have been moved to another project or have left the company (although I think that this code has been at least partially subcontracted to offshore companies), the new developers who are in charge of maintaining the program are very often unable to understand the existing code, especially if it has not been properly documented.

    I'm pretty sure that this is the reason why so many "old" bugs have not been (and will never be) fixed. They just can't. They are out of control or are not skilled enough. Otherwise, this would demonstrate that they just don't care, which wouldn't be a surprise either.

    --Patrick
    F
    Community Manager
    FeedbackCommunityMemberCommunity Manager
    Community Manager
    July 9, 2021

    Now one year later this is still a problem. I have a little addendum to the problem. This is actually much much much worse when the actual files resides on a mounted network drive.

    But why in the first place relay on timestamps to determine when files have been changed - silly. Use a MD5 sum instead. Just MD5 sum the file - save the MD5-sum in the database and when determine if file is changed - do an MD5 sum and compare to the saved in the DB. Thats all there is to it...used this method a lot for various development projects.

    The catch about timestamps is that most backup solution "touch" the file when doing backup, silly i know, but thats often the way they do it - this means after each backup everything is changed. This forces me to have the files in an "unbackuped" space - and after changing stuff they will be rsync'ed to a backed up space - requireing double internal storage. I know disk space is cheap, but still a problematic way of doing things....

    S
    SamoreenAuthor
    Inspiring
    April 21, 2020
    After multiple tests, I confirm. Fixing the wrong state of the metadata status arrow by using the Ctrl+S | Read metadata from file sequence is no longer working while in the Library module. However, when switching to the Develop module, the status arrow is reset.

    So, not only that old bug is still not fixed but it just became worse.

    Adobe, please tell the programmer who is in charge of this to keep his/her hands off that code.
    --Patrick
    S
    SamoreenAuthor
    Inspiring
    April 20, 2020
    Hi,

    There's a new related bug with version 9.2.

    Previously, when using the Metadata | Read metadata from file command, the up/down arrow at the top right corner of the thumbnail was reset (which is expected). Now it shows a down arrow, meaning that after reading the metadata from the XMP  file, the catalog is still considered as not in sync with the XMP file.

    The original bug is already annoying enough, it's not necessary to add insult to injury.
    --Patrick
    F
    Community Manager
    FeedbackCommunityMemberCommunity Manager
    Community Manager
    April 1, 2020
    Hello, I just wanted to say that I experience the bug for many years and now I finally found the exact description of reason why Lightroom wrongly behaves. I was getting mad after all the cmd+s and still seeing "metadata has been changed". I desperately await once there is a comparable application to Lightroom so I can move away from it for good. Since it takes more 10 years to fix such a small bug to Adobe, I believe that another company (hopefully Affinity) will introduce their solution sooner. 
    S
    SamoreenAuthor
    Inspiring
    February 15, 2020
    Still not fixed in version 9.2. Are you surprised ? I'm not. Lightroom is like Arthur Rimbaud's Drunken Boat.

    As I was going down impassive Rivers,
    I no longer felt myself guided by haulers:
    Yelping redskins had taken them as targets
    And had nailed them naked to colored stakes.
    --Patrick
    S
    SamoreenAuthor
    Inspiring
    January 1, 2020
    Michael is correct,

    The problem is that the timestamp recorded in the catalog is not kept in sync with the time stamp recorded in the XMP file, as explained at the beginning of this thread. I was just trying to make this clear to the people who should have fixed this since many years. Since the technical explanations that have been given by me and John + the test allowing to reproduce the problem seem to be too complicated for them, I was trying to give a more simplistic way of tracking and reproducing the bug.

    Also, I should say that I have been contacted by people from India who seem to be in charge of this case. The questions they asked and the tests that they asked me to do have demonstrated that they merely didn't understand what has been explained and reported in this thread. Anyway, they have stopped answering my questions.
    --Patrick
    M
    MBeckerAC
    Participating Frequently
    January 1, 2020
    The problem is not keeping the xmp sidecars up to date - but the "meta data changed flag" is resulting in re-uploading the images to the publication services.

    Re-uploading as a result of just and only touching (i.e. viewing) an image is not only superfluous, but a serios performance problem. For me, if I am viewing images, this results in 100s, sometimes 1000s of images to be re-uploaded

    Another aspect:
    If the touched image is still in the preview cache ... then the "meta data changed" flag ist *not* set.

    So, it looks like touch time is not the reason and not an explanation for this software bug.
    Show more replies
    Terms & ConditionsAccessibility statement
    Using the community Terms of use Privacy Cookie preferences Do not sell or share my personal information ad choicesAdChoices