Lightroom Classic 9.0 - Build previews is also updating images and sidecars

Report · Nov 11, 2019

Lightroom Classic 9.0 , Camera Raw 12.0, macOS 10.13.6 - High Sierra, all patches applied

The command "Library -> Previews -> Build Standard previews" is not only building previews it is also updating many, but not all, of the images referenced in the catalog.

I have 'Automatically write changes into XMP' checked - metadata changes for all images in the catalog (c53,500) were up to date befoe the Lightroom upgrade.

I recently upgraded to Lightroom Classic 9.0. As a result of all the file renaming that happened in the upgrade I decided to start again with the previews, after all just processor time. How wrong...

When I started building previews, large numbers of files were rewritten. I noticed this because first, Time Machine started producing large backups, and then this behaviour was replicated by my other backup software.

When I stopped building previews, the file updates stopped, only to restart next time I started building previews.

A substantal proportion of the images (for jpg and Tiff) or sidecars (xmp for raw files) were processed. There is considerable variability; some folders it is hardly any, in others it would be every file.

So, why were these files changes done when the preview was generated? If the metatdata written to file was not correct in the earlier version then surely the correct mechanism for handling this should be to use the metadata status flags and let me choose what to do.

Report · Nov 11, 2019

Sounds like this has become unexpectedly serious and very frustrating for you. I hope we can help sort this out . . .

"I recently upgraded to Lightroom Classic 9.0. As a result of all the file renaming that happened in the upgrade I decided to start again with the previews, after all just processor time. How wrong..."

Can you shed a little more light on this?

What was the exact nature file naming that took place after the upgrade?

Thanks

Report · Nov 11, 2019

Renaming the catalog by appending -2 without properly sorting out the preview file. From memory, the old one was c22GB and the new one not even a tenth of that, maybe much less.

Done and dusted, I deleted the old and 'migrated' preview file and created a new one. Hence the rebuild of the previews for all the images. Only took two days.

I'm more annoyed about the 70+ GB of jpg, tif and xmp files that appeared in my backups as a result of doing nothing but creating previews but thats history as well. See the final paragraph of my original post

Report · Nov 11, 2019

LR 9 has introduced a new Catalog format and a new Process Version. First interaction of any sort with an image, causes it to adopt the brand new PV (apart from, when its currently set PV is not a recent one).

The default folder location of previews will naturally have changed when the LR9 Catalog adopts a different name than the prior Catalog had. Because prior previews should be transferred across (assuming default behaviour) it then becomes a matter of how long those prior previews can remain valid.

It appears that since generating a new Preview needs LR to access the image and apply current Develop processing, that counts as a fresh 'interaction with' the image so far as the Catalog is concerned, hence triggers an update to new PV, and this change triggers a metadata write.

Should be a one-off, if this does prove to be the explanation.

Report · Nov 11, 2019

Its not all images, just a selection get rewritten.

The generation of previews hasn't changed the process version. I just ran 'Find previous Process Photos' from the menu - results in c44k of c53k, a bit lower than I expected but in the right ball-park. Plus I can't see anything in the release notes about a new process, but then it doesn't say new catalog format either so thats not conclusive.

Taking one folder of very recent but pre-update photos and running through the first half dozen or so, it seems that it may be as a result of some develop module changes having been previously applied, but not all changes result in a rewrite:

White balance/colour get rewritten
Just perspective change and nothing else get left alone
Just lens adjustment get left alone

Anyway, it is what it is; I posted this largely as a heads up incase anyone else was perplexed as to why their backups suddenly went crazy.

Report · Nov 12, 2019

I'm now pretty sure that it is related in part to the change to Camera Raw 12.0 and cropping images.

It seems that for many images, Camera Raw 12.0 is nudging the crop slightly when the preview is generated and Lighroom is immediately recording the change either in the file (jpg, tif) or the associated xmp sidecar. I suspect that Camera Raw may be making other changes as well.

This should have been handled in the same way that LR handles its own process changes - flag the difference and let the user decide what to do.

I looked at three files where the xmp had been updated by generating the previews. Other than recording date/time and vesion changes only the crop data is different.

These are the diffferences in XMP files; for each, pre-upgrade is shown first then post-upgrade

First image

53c53

< xmp:MetadataDate="2019-07-28T07:31:23+01:00"

---

> xmp:MetadataDate="2019-11-10T18:21:38Z"

65,68c65,68

< xmpMM:InstanceID="xmp.iid:3d0231ab-1c21-441f-9859-508cea893def"

< xmpMM:DocumentID="xmp.did:3d0231ab-1c21-441f-9859-508cea893def"

< xmpMM:OriginalDocumentID="xmp.did:3d0231ab-1c21-441f-9859-508cea893def"

< crs:Version="11.3"

---

> xmpMM:InstanceID="xmp.iid:0443d7ad-1a54-40b2-9a14-023715479755"

> xmpMM:DocumentID="xmp.did:0443d7ad-1a54-40b2-9a14-023715479755"

> xmpMM:OriginalDocumentID="xmp.did:0443d7ad-1a54-40b2-9a14-023715479755"

> crs:Version="12.0"

181,184c181,184

< crs:CropTop="0.027127"

< crs:CropLeft="0.027127"

< crs:CropBottom="0.972871"

< crs:CropRight="0.972871"

---

> crs:CropTop="0.027115"

> crs:CropLeft="0.027115"

> crs:CropBottom="0.972845"

> crs:CropRight="0.972845"

211,213c211,213

< stEvt:instanceID="xmp.iid:3d0231ab-1c21-441f-9859-508cea893def"

< stEvt:when="2019-07-28T07:31:23+01:00"

< stEvt:softwareAgent="Adobe Photoshop Lightroom Classic 8.3.1 (Macintosh)"

---

> stEvt:instanceID="xmp.iid:0443d7ad-1a54-40b2-9a14-023715479755"

> stEvt:when="2019-11-10T18:21:38Z"

> stEvt:softwareAgent="Adobe Photoshop Lightroom Classic 9.0 (Macintosh)"

310c310

< crs:Version="11.3"

---

> crs:Version="12.0"

Second Image

53c53

< xmp:MetadataDate="2019-07-27T10:57:22+01:00"

---

> xmp:MetadataDate="2019-11-10T18:22:12Z"

65,68c65,68

< xmpMM:InstanceID="xmp.iid:9c67d085-27fe-4f0f-b834-8897ccce9054"

< xmpMM:DocumentID="xmp.did:9c67d085-27fe-4f0f-b834-8897ccce9054"

< xmpMM:OriginalDocumentID="xmp.did:9c67d085-27fe-4f0f-b834-8897ccce9054"

< crs:Version="11.3"

---

> xmpMM:InstanceID="xmp.iid:8db72f92-36d7-4356-89d6-956faf4f82e1"

> xmpMM:DocumentID="xmp.did:8db72f92-36d7-4356-89d6-956faf4f82e1"

> xmpMM:OriginalDocumentID="xmp.did:8db72f92-36d7-4356-89d6-956faf4f82e1"

> crs:Version="12.0"

186,189c186,189

< crs:CropTop="0.047808"

< crs:CropLeft="0.047807"

< crs:CropBottom="0.952193"

< crs:CropRight="0.952192"

---

> crs:CropTop="0.047832"

> crs:CropLeft="0.047809"

> crs:CropBottom="0.952191"

> crs:CropRight="0.952168"

216,218c216,218

< stEvt:instanceID="xmp.iid:9c67d085-27fe-4f0f-b834-8897ccce9054"

< stEvt:when="2019-07-27T10:57:22+01:00"

< stEvt:softwareAgent="Adobe Photoshop Lightroom Classic 8.3.1 (Macintosh)"

---

> stEvt:instanceID="xmp.iid:8db72f92-36d7-4356-89d6-956faf4f82e1"

> stEvt:when="2019-11-10T18:22:12Z"

> stEvt:softwareAgent="Adobe Photoshop Lightroom Classic 9.0 (Macintosh)"

315c315

< crs:Version="11.3"

---

> crs:Version="12.0"

Third image

53c53

< xmp:MetadataDate="2019-07-27T10:58:19+01:00"

---

> xmp:MetadataDate="2019-11-10T18:22:15Z"

65,68c65,68

< xmpMM:InstanceID="xmp.iid:917e8caa-b52a-4a5f-9ed2-3e1ab4d8f8e1"

< xmpMM:DocumentID="xmp.did:917e8caa-b52a-4a5f-9ed2-3e1ab4d8f8e1"

< xmpMM:OriginalDocumentID="xmp.did:917e8caa-b52a-4a5f-9ed2-3e1ab4d8f8e1"

< crs:Version="11.3"

---

> xmpMM:InstanceID="xmp.iid:10e9f57c-1dec-4007-9bc7-a1c76afe3152"

> xmpMM:DocumentID="xmp.did:10e9f57c-1dec-4007-9bc7-a1c76afe3152"

> xmpMM:OriginalDocumentID="xmp.did:10e9f57c-1dec-4007-9bc7-a1c76afe3152"

> crs:Version="12.0"

186,189c186,189

< crs:CropTop="0.016941"

< crs:CropLeft="0.016919"

< crs:CropBottom="0.983075"

< crs:CropRight="0.983052"

---

> crs:CropTop="0.016959"

> crs:CropLeft="0.016911"

> crs:CropBottom="0.983083"

> crs:CropRight="0.983034"

216,218c216,218

< stEvt:instanceID="xmp.iid:917e8caa-b52a-4a5f-9ed2-3e1ab4d8f8e1"

< stEvt:when="2019-07-27T10:58:19+01:00"

< stEvt:softwareAgent="Adobe Photoshop Lightroom Classic 8.3.1 (Macintosh)"

---

> stEvt:instanceID="xmp.iid:10e9f57c-1dec-4007-9bc7-a1c76afe3152"

> stEvt:when="2019-11-10T18:22:15Z"

> stEvt:softwareAgent="Adobe Photoshop Lightroom Classic 9.0 (Macintosh)"

315c315

< crs:Version="11.3"

---

> crs:Version="12.0"

Report · Nov 12, 2019

Good spot. More of a practical impact, I suppose, for DNG / JPG than for proprietary Raw: if a full image file is updated by each metadata write, and not just a sidecar. Of course such an update would be happening anyway, at some point, for those images that do get actively revisited.

Report · Nov 12, 2019

I suspect the crop change is very small, on the few I looked at its in the fourth digit, but I'm not sure what the units are. But, there may be other changes as well.

Short of looking through my backups to work out which several thousand images changed I have no idea which files they were
Its not easy to extract the data from jpg, tif images to see what the changes were
Sure I could browse the image and see if it looks grotesque but that would take ages, find the file name in the backup, navigate to that image in LR, ...
Some of those changed are over 600 meg which is how I noticed that Time Machine was going beserk.

They have a mechanism for handling changes like this and didn't use it - ignorance, oversight, lazy...?

Report · Nov 19, 2019

One thing the updates also did was change the Metadata Date so using a plug-in I managed to find all the images that had been modifed as a result of publishing previews: in round numbers 9,000 out of 55,000.

Still no consistency that I could see on why the image was changed, e.g. most had some cropping but many didn't.

I even found one where the development history consisted of three items all of these being actions ages ago so not related to V9.0:

Import
Update to the then latest process
Publish

For the selection of images I looked at, there is nothing in the history resulting from the change made by creating a preview.

Report · Nov 21, 2019

I found one Tif image that was rewritten because I had removed a keyword in LR 8.4. Looking at the backup from immediately prior to the upgrade the xmp embedded in the file was wrong, it still contained the word I had removed some months earlier.

For some reason, creating the previews recreated the xmp. It then noted there was a difference and wrote the new xmp into the file.

So it looks like some of the changed files are as a result of errors/bugs in earlier versions being surfaced.