P: Still inconsistent capture date/time for photos and videos

[John R Ellis]

Update 5/17/2018: Though LR 7.1 made improvements, LR 7.3.1 still has two closely related problems with a single underlying cause:

- With photos and videos missing metadata capture date/times (e.g. scans), there is still an inconsistency between the times shown under the thumbnails in grid view and in the Metadata panel and the hidden, internal times used for sorting in grid view.

- Changing IPTC Date Created in the Metadata panel, either by editing the field or using Metadata > Copy/Paste Metadata, similarly causes inconsistent values to be shown and sorting and searching to work inconsistently.  It also causes date metadata to be written back to the files that doesn't conform with the Metadata Working Group's standard.

The underlying cause is architectural: LR doesn't have a single internal catalog field representing "capture time".  Rather, it maintains capture time in several different fields, and the various parts of LR update those fields inconsistently.

See here for precise recipes to replicate these bugs:

https://feedback.photoshop.com/photoshop_family/topics/lightroom-still-inconsistent-capture-date-tim...

https://feedback.photoshop.com/photoshop_family/topics/lightroom-still-inconsistent-capture-date-tim...

See here for a workaround: 

https://feedback.photoshop.com/photoshop_family/topics/lightroom-still-inconsistent-capture-date-tim....  

-------------

LR 5.7 still shows inconsistent capture date/times for videos. For a test .avi on Windows 8.1, the date in grid view appears to be the file system's last-modified time, while Capture Date/Time is set to the time of import.This problem was declared fixed in LR 5.5, and it appears to have been fixed for images, but not videos:http://feedback.photoshop.com/photoshop_family/topics/inconsistent_dates_for_files_missing_date_time...I'm opening a new topic, since the previous one has been marked "Solved". Untitled2_inline-402b28f5-7fce-4485-bbc7-1eba2e9ee1ed-622028544.png Untitled_inline-a0d82271-2553-4a54-8824-20de68939af3-906850541.png

[John R Ellis]

LR 7.0.1 doesn't fix the outsanding bug with editing IPTC > Date Created in the Metadata panel or using Metadata Copy/Paste Metadata to copy IPTC > Date Created: https://feedback.photoshop.com/photoshop_family/topics/lightroom-still-inconsistent-capture-date-tim...

[John R Ellis]

This bug was fixed in LR 7, but see here for a related bug that was introduced: https://feedback.photoshop.com/photoshop_family/topics/lightroom-still-inconsistent-capture-date-tim...

[abhishek_india]

We are able to reproduce the inconsistency observed in the capture date/time for the images attached by John. We have logged a new bug for the same. 

[John R Ellis]

Good, thanks.

[Sunil_Bhaskaran]

Please see Lightroom Classic 7.1. This issue is fixed.

Thanks,
Sunil

[courtneycastle]

Not sure if my issue is the same as others who have posted here, but I'm dealing with a large-scale nightmare relating to creation dates in my LR catalog of 20K+ images. I recently made a lot of keyword updates and saved the metadata to the files. Then I noticed that for any of my photos that didn't have embedded capture/digitized metadata, the "Metadata Date" was changed to the day I saved the keyword metadata to the files. These files now only show the new date in the metadata since they lack capture dates. 

However, in Library grid view, the thumbnails still display the correct creation dates even though in the metadata panel all date fields are empty except for Metadata Date. So when I followed the advice of someone above to save metadata to file and then read metadata from file, I ended up wiping the correct capture date that had been displayed on the thumbnail, replacing it with the date I last edited the metadata.

If there is no way to reset the capture times to those saved in the LR catalog (i.e. the ones displayed on the thumbnails), then I'm facing hundreds of hours of manually correcting the metadata. In other words, nightmare.

I have my entire LR library synced/backed up on Dropbox. It is possible to restore a previous version from Dropbox, but this is only possible on a file-by-file basis. I'm now regretting not backing up my library to my Time Machine.

[Mikkelikski]

Maybe it is obvious that this will not work, but could you restore a previous version of the catalog? Before you made the edits? 

Im horrified by the thought of your situation. Hope you encounter a fix!

[courtneycastle]

I could, but that would not solve the problem because LR wrote the metadata to the original files.

[John R Ellis]

Not the same issue as the topic in which you posted.

Please reference the new conversation here: Lightroom: Recovering file last-modified date/time?

[axmelissarios1]

Hi all,

Just to reiterate what Sunil said in a comment above, this issue should be fixed in Lightroom Classic CC 7.1. Please let us know if you're still seeing this issue after updating.

Thanks,
Melissa

[ItWasNotMe]

1. Did you fix ALL the bugs referenced in this thread or just the one referenced in the title? You will see many comments (i.e. complaints), including mine, that disparate errors have been merged into this thread.
2. Anyone who has used Edit Capture Time and/or Paste Metadata will have different values in different fields in the catalog for the relevant images. Where is the utility to make these consistent?

[Victoria Bampton LR Queen]

It's impossible for them to promise they've fixed every single one Stephen, because even if they think they've fixed everything, someone might still have a problem. If you can still reproduce any of the issues you've noted, please feel free to update the thread or start a new one for the specific error you're still seeing.

[ItWasNotMe]

1. I did start a specific thread, it got merged into this
2. It would be quite straightforward for them to enumerate all the errors in this thread and answer yes/no - if you read the thread you will see I suggested this earlier
3. Its my job to USE the software not test it. Remember I'm paying them, not the other way round

[John R Ellis]

Sunil and Melissa,

In August, Adobe asked me to provide recipes for the issues in this thread. I provided two such recipes, updated the first post in this topic with them, and communicated that by email. After 7.0.1, Adobe asked me to test, which I did, I updated one of the recipes, and posted that the other one hadn't yet been addressed.

In LR 7.1, the recipe concerning photos missing metadata capture date/times has been completely addressed, while the recipe concerning edits to IPTC:DateCreated has not.

Details:

This topic concerns two ways to provoke the underlying architectural inconsistency in the capture date/time fields, in which the values shown under the thumbnail, in Metadata > Default, Metadata > EXIF, and Metadata > IPTC would be inconsistent, and different parts of the application would use different values (sorting, file renaming, the Library filter bar, smart collections).

1. Import a photo or video with missing metadata date/time fields or with just EXIF:DateTime but not EXIF:DateTimeOriginal (e.g. importing scans or from very old digital cameras):

https://feedback.photoshop.com/photoshop_family/topics/lightroom-still-inconsistent-capture-date-tim...

I've tested this recipe and tested a similar one for video, and it appears to be finally fixed!   There are no inconsistencies between the dates shown under the thumbnail and in Metadata > Default and Metadata > EXIF.  Sorting thumbnails in grid view and the Library Metadata filter bar are consistent with those values, and renaming files in grid view and in export uses those values consistently.  

2. Change IPTC:DateCreated with with the Metadata > IPTC panel or by copying and pasting that field with Metadata > Copy/Paste Metadata.  

https://feedback.photoshop.com/photoshop_family/topics/lightroom-still-inconsistent-capture-date-tim...

Though the Metadata Working Group specification to which LR subscribes says that IPTC:DateCreated should be consistent with EXIF:DateTimeOriginal and XMP:DateCreted, changing the IPTC:DateCreated in the Metadata panel doesn't update the date/times shown under the thumbnail, Metadata > Default, or Metadata > EXIF.

The previous workaround continues to work: Select all the affected photos and doing Metadata > Edit Capture Time and then Adjust All will set IPTC:DateCreated to the same value as shown in Metadata > Default and Metadata > EXIF.

[John R Ellis]

"Anyone who has used Edit Capture Time and/or Paste Metadata will have different values in different fields in the catalog for the relevant images. Where is the utility to make these consistent?"

In my testing, the previous workaround continues to work: Select all the affected photos, do Metadata > Edit Capture Time, and click Adjust All.  That  will set IPTC:DateCreated to the same capture date value as shown in Metadata > Default,  Metadata > EXIF, and under the thumbnail.

[ItWasNotMe]

They may look the same on screen but in the underlying database they are still different. 
For example, should you use a plug-in then you are at the mercy of the gods as to which of the many fields the developer has chosen.
[Edit, see your own comment below on copy/paste. Until that is corrected, in any event; any utility would be of limited use ]

[John R Ellis]

"should you use a plug-in then you are at the mercy of the gods as to which of the many fields the developer has chosen."

The LR SDK API appears to reflect accurately what is shown in the application to the user.  All photos have a "captureTime" that is the same as what's shown in the app, and photos that have EXIF:DateTimeOriginal in their metadata have "dateTimeOriginal".

{--table: 1
    [LrPhoto( id "3875" )] = {--table: 2
        captureTime = 461908947, 
        date = "2015-08-22T04:02:27", 
        path = "/Users/john/Downloads/capture-time-bug copy/test2.jpg"}, 
    [LrPhoto( id "3874" )] = {--table: 3
        captureTime = 493531347, 
        date = "2016-08-22T04:02:27", 
        dateTimeOriginal = 493531347, 
        path = "/Users/john/Downloads/capture-time-bug copy/test1.jpg"}, 
    [LrPhoto( id "3876" )] = {--table: 4
        captureTime = 556603347, 
        date = "2018-08-22T04:02:27", 
        dateTimeOriginal = 556603347, 
        path = "/Users/john/Downloads/capture-time-bug copy/test3.jpg"}, 
    [LrPhoto( id "3877" )] = {--table: 5
        captureTime = 532143432, 
        date = "2017-11-12T01:37:12", 
        path = "/Users/john/Downloads/capture-time-bug copy/test4.jpg"}}

[Sunil_Bhaskaran]

Thanks a lot for the details, John.
We are looking into the second issue you mentioned.

Thanks,
Sunil

[ItWasNotMe]

Fortune favours the lucky then...

 Lets conduct a little thought experiment. Conjecture:

• There is a field captureTime in the Database table, but not all of the code uses this (maybe its seen as slow to access, who knows)
  
• There are two fields in the XMP, exif:DateTimeOriginal and photoshop:DateCreated,  (in reality there are more but twos enough for this example) and these are used frequently in the code but not consistently, which is the symptom we see

 We know that ‘Copy/Paste’ metadata and ‘Edit Capture Time’ don’t update consistently, so

• Lets say ‘Edit’ updates all three
• Lets say ‘Copy’ only updates photoshop:DateCreated and (maybe) captureTime but not exif:DateTimeOriginal
• So, we have some images for which these two xmp values may differ 

Now, there are lots of places that capture date is used.

• Several points in the library display (i.e. the inconsistencies in the title of this thread)
• Sort by
• The API
• The edit and copy functions themselves need to display something as a starting point
• And so on

There is no guarantee that all of these use captureTime, or exif:DateTimeOriginal. From you example they can't, otherwise we would not see variability. In V6 for example some of the fields had quite complex conditional statements trying to find a date that could be displayed.

Lets have Adobe ‘fix’ the immediate problem and have the ‘Copy’ write exif:DateTimeOriginal as well (and captureTime if it doesn't). Further, it doesn't standardise the code to use the value in the database rather than the XMP and doesn’t provide any utility function to align these values for each image. 

Were this the case, then

• At any point in the future, any of its developers may decide that for the little fragment of code they are changing, it would be better to use the ‘other’ field.
• Then for some data stored prior to the ‘fix’, we are back to the inconsistency and start all over again. This is why it should repair the database as well as fixing the code.

 There is a reason that camera supports have three legs and not four – stability.

 For the same reason its poor practice to have the same data value in more than one place.

[ItWasNotMe]

This is what I think is happening

On a test catalog  I took three images created in 2010, 2014, 2017 from a supported camera

 ·      2010 image, I left unchanged

·      2014 image, I pasted the date from the 2010 image

·      2017 image, I used Metadata -> Edit Capture time; set to random value

First difference:

Paste metadata doesn’t write to the field captureTime in Table Adobe_Images

Action, Copy/Paste from 2010 image to the 2014 image

• Before "2014-04-25T14:09:38.24", After "2014-04-25T14:09:38.24"

Whereas Metadata -> Edit Capture time does, on the 2017 Image

• Before, "2017-03-28T11:53:38.41, After "1995-01-01T00:00:01"

Second difference:

Only image related xmp date field that I could see copy/paste changing is photoshop:DateCreated

These are selected changes on the Copy/Paste xmp, three of the values are unchanged from their original 2014 values (created in import)

• exif:DateTimeOriginal="2014-04-25T14:09:38.24"
• xmp:ModifyDate="2014-04-25T14:09:38.24"
• xmp:CreateDate="2014-04-25T14:09:38.24"
• photoshop:DateCreated="2010-01-01T12:22:19.73

Whereas the Edit Capture Time changes an additional field, exif:DateTimeOriginal

• exif:DateTimeOriginal="1995-01-01T00:00:01"
• xmp:ModifyDate="2017-03-28T11:53:38.41"
• xmp:CreateDate="2017-03-28T11:53:38.41"
• photoshop:DateCreated="1995-01-01T00:00:01" 

Note, neither action changes the xmp:CreateDate, or xmp:ModifyDate both are the time that I shot the relevant image.

[Edit] Forgot to say there are other date fields in the tables that I didn't look at, such as harvested date which may or may not have been updated.

[John R Ellis]

I agree that the current architecture is very fragile.   Some background: I believe that it was originally motivated by a desire to have the database metadata tables reflect the contents of the corresponding date/time metadata fields. (There are four different photo metadata standards with at least 12 different date/time fields.)  Several years after LR was released, the Metadata Working Group came out with its revised standard, which LR adopted.  That standard introduced some significant changes in the way that the multiple date/time metadata fields should be handled.   

It would be straightforward for LR to have its cake and eat it too, maintaining the original metadata fields in the database but have a single unified method for accessing "capture time" (whether via a function or a database field).  But somewhere along the way the engineering lost its way on this.

[ItWasNotMe]

Yep, its often referred to as data abstraction, dates back to before 1980 as a concept.

Sadly, though, I think swathes of the code don’t use ‘the database’ per se, rather they parse the xmp string. This is always stored as a composite string in one of the columns. For example, if my analysis below is correct, then

• ‘Edit Capture Time’ doesn’t take the ‘true’ database field as the source
• Reminder, Paste didn’t set this field, therefore if ‘Edit’ used this as the source then it would revert the date back to the value before the paste, rather than ‘correcting’ the issue
• Conclusion, ‘Edit’ must be taking photoshop:DateCreated as its source
• This is only in the xmp

 Another rather old concept is refactoring, Wiki suggests first known use of the term is 1990.

 In this context, you might refactor the code and make all the ‘read’ occurrences of capture date use the 'true' database field and apply data abstraction to the write occurrences.            Removes some inconsistency but highlights another problem, the 'true' database field can be incorrect.

[ItWasNotMe]

Missing from the checklist at the top of this thread, and still wrong in the 7.1 release, is that when you sort by capture date in the grid library views the sort order is not always the same order as you would expect from the dates displayed in the 'Capture Date' field in the default metadata view. 
[Edit - revised to state not always]

[ItWasNotMe]

Still not completely fixed in 7.2 on MacOS 10.13.3; see posts below for details.

[LauraL982001]

I was hopeful that Lightroom Classic CC 7.1 would fix this issue, but it didn't.  I have updated to 7.2 and still have the same problem, but only with iphone 6s plus videos (.MOV).  

When I import videos and images I like to rename all my files to include the capture date and time (YYY-MM-DD Hour.Minute.Second).  My 5d Mark iv images and videos (.MOV) import and rename based on time perfectly and it matches all the metadata times.  Same with iphone images.  However, the time in the iphone video filenames are ALWAYS wrong, and I can't for the life of me figure out where the hour even comes from.  It's always 5 or 6 hours off, which I suspect changes based on whether it's daylight savings or not.  As a result of the incorrect time, any videos taken after 6 or 7 pm, also have an incorrect date.

To be clear, I am expecting a filename based on the time CREATED, just like images and other videos I import.  

Example 1-iphone 6s plus:
Filename time: 14.22.28 (=2:22:28 pm; 6 hours off when daylight savings ends)
Date Time Digitized: 8:22:28 AM (correct time of capture)
Date Time: 10:37:17 PM (correct time of import)

Example 2-iphone 6s plus:
Filename time: 23.43.57 (=11:43:57 pm; 5 hours off during daylight savings)
Date Time Digitized: 6:43:57 PM (correct time of capture)
Date Time: 9:48:41 PM (correct time of import)

At first glance I might have said it was a time zone error, except the metadata is correct in the videos and all other images and my professional camera videos renamed with the same filename structure are always correct upon import, with the filename date/time matching all metadata.  

Observations:
*Images have 3 date/time areas; all seem to match; all are the exact date/time created:
1) Date Time Original     =date/time created
2) Date Time Digitized    =date/time created
3) Date Time                      =date/time created

*However, Videos only have 2 & 3 of the above time areas where:
2) Date Time Digitized   =date/time created
3) Date Time                     = date/time IMPORTED (WHY anyone needs this is beyond me, but whatever, even this doesn't match the random time in my iphone video filenames)

*Sorting images/videos by date created works correctly-even when the filename shows an incorrect time.  I presume this is because the metadata is correct, regardless of the filename.

*Auto importing the exact same iphone videos as above  from a watched folder and using the exact same naming structure (date hour.minute.second) works correctly.  I've been using this workaround for years.  But it is full of flaws: Lightroom will re-import the same files-with the exact same metadata- if air dropped to the watched folder by accident. And it is a pain to figure out what has and hasn't been airdropped if done out of order, so my catalog becomes full of duplicate images and videos that have a -2 at the end of the filename).  

This is a HUGE problem.  My workaround is flawed and it causes me to not back up my iphone images as often as I should.  Even when I don't import using the hour.minute.second filename structure, the date is off for all iphone videos taken after 6pm or before 6am, which is UNACCEPTABLE for organizational purposes.  Ultimately, I can't believe this has been going on for years and people just accept it.  I pay every month and I haven't been able to import videos with the correct time.  That's basic.  FIX IT.   

Note:  I have also tried renaming on import with only the date and without the time, but this still results in incorrect dates for all videos taken after 6 or 7 pm.  Not renaming with dates is not an option.