Lightroom Catalog, nothing inside since 04-2018!

[timballic]

About 10 days ago, after updating to ver 9.03. I reformatted my Lightroom Classic Catalog as requested, (I'm using iMac with OS 10.15.15 Catalina.)

Lightroom warned me to ensure I had a recent backup version before reformatting. So I backed up, (it took about 10-15 mins, as usual), and I checked the resultant backup file, which was named "14-07-2020". All seemed OK. There it was.

So I reformatted the catalog and was told to reload from the latest version backup.

I loaded that 14-07-2020 file and on Lightroom opening, there was nothing more recent than 18-04-2018!

There should be many thousands of files later than that, right up to 14-07-2020!

  I back up the catalog nearly every time I quit Lightroom, using manual command, rather than automatic, (about 4 times a week). The blue line appears, and after 10-15 mins it's finished. It's never indicated that it hasn't completed all the backup.

So I assumed I had loads of recent backups to select.

I tried several. All stop at 18-04-2018! Nothing more recent.

That date of 18-04-2018, rang a bell, it was just about when I moved to Lr Classic Subscription model.

What's happened?  Is it as hopeless as it appears to be?

[JP Hess]

Have you been backing up since you started the subscription model? I don't understand why you thinking you were instructed to reload your last backup after your catalog was updated. After the update was completed Lightroom Classic should have simply opened that updated catalog and continued by using that new updated catalog. The only time you should have to use a backup is when/if the working catalog fails. Is there another catalog on your system that has a -2 as part of its name? If there is then double-click on that catalog and see if that solves your problem.

[timballic]

Thanks. I don't know why it told me to either. I'd expected it to just
start up as usual, but it didn't.
I've checked all the 2.2 backups (in fact I've checked everything) and all
finish at 18.04/2018.

[timballic]

..and Yes, I've been backing up, as I thought, 3-4 times a week, since going with the subscription model.

 I've just looked in my external hard drive Lr Backups, (Which I make infrequently, by changing the location setting for the Backup), and there, the  2.2 files are present up to March 2020!

I don't know why backups to my EHD have worked, and those to my iMac HD haven't! That makes it even more strange.

Sadly I closed that EHD down at end of March as it kept on spontaneously unmounting, I've only just replaced it, since this problem with the  Lr catalog

So, it's all my work during lockdown that's gone. About 500+ hours 😞

[timballic]

Just rechecked. That date of 18-04-2018 doesn't coincide with the change to the subscription Version, which I made in Jan 1919. So even before going with the subscription model, the backups to my iMac HD weren't happening as they should have, but to the EHD they were. 

If only that hadn't stopped this March. 😞

[JP Hess]

When that dialog appears prompting you to backup your catalog, look at it closely next time and pay particular attention to the folder path where the backup is being saved. That has arbitrarily changed on me a couple of times, and the backup has been saved to a path that I wasn't expecting. I'm thinking that perhaps this might have happened in your case.

It is possible to go to your catalog settings and choose the option to backup the catalog the next time Lightroom exits regardless of how often you have normally chosen to have your catalog backed up. That way you can quickly see the dialog and examine that path.

[timballic]

Thanks Jim, If that has happened, I haven't yet discovered their location. However, I don't think it can be so, as the files have the correct dates, when made.

[timballic]

Backing up catalog now seems to be working fine. So the problem occured before the reformatting. And, as I explain above your last answer, it only occured when copying to the iMac HD, not when I relocated the backups to the EHD, which I only occassionaly did. (Which sadly I haven't done since March, due to problems with MacOS Catalina, not playing well with the EHD!)

[JP Hess]

I know nothing about the Mac, so I won't try to offer anymore advice. I sincerely hope you are able to find a more recent back up.

[timballic]

Many thanks Jim. (I don't think this will be a Mac thing though.)

I realise, that having left the problem for two weeks, before asking for help, I'd forgotten the sequence of events, and vitally the point that confused you.

After starting the  Catalog Upgrade, LrC DID reopen,  BUT only with files up to 18.04.2018 visible, ie everything from then up to 14.07.2020 apparently missing. . THEN I tried restarting from the last backup, and got the same result.

Now I'm wondering if it did in fact copy the "missing" files (18.04.2018 - 14.07.2020), but has put them somewhere else, maybe even created another catalog for them? (My original single catalog was very large with everything from 2007 in it.) If this is a possibility, where should I begin to look for them?

[timballic]

Jim, They're found. ;o)

Following instructions below,(dj_paige) I searched my Mac for all .LRCAT files, until I found one complete!

I would swear it was the very one I tried to reload from after LrC reopened with 2+ years files missing!? Anyway, they're all on LrC now.

What I'd love to know, is what's been going on, and perhaps more imporantly, how can I prevent it from happening again?! How can I ensure that backup really is backing up everything from now on? 

[dj_paige]

"What I'd love to know, is what's been going on, and perhaps more imporantly, how can I prevent it from happening again?! How can I ensure that backup really is backing up everything from now on?"

This is pure speculation because I was not sitting at your computer when all this happened. But it does seem to explain the facts you presented.

At some point (around April 18, 2018), the catalog you were using was switched, and you began using a catalog in a different location, perhaps because you opened a recent backup. So the first catalog stopped getting used. Then, at some point, Lightroom looks for a catalog in a default location and finds the one that hasn't been used since around April 18, 2018, and there are no photos since that date. The work you did after that date is in a different catalog.

[Just Shoot Me]

You keep saying "I ReFormatted my LrC Catalog"??? You do not do that. There is NO option to format a LrC catalog.

There is options to FORMAT a Hard Drive connected to your system. But nothing like that for the LrC Catalog File which is Just a Database file, A SINGLE File stored someplace on your computer.

So Please Stop using the word Format or Reformat. It makes helping you very confusing.

To start over, start Fresh with LrC, all you have to do is Start, That Is START (Create), a New Catalog. No need to Delete whatever is recorded in your original LrC catalog or delete any images from your computers hard drive.

[timballic]

I do apologise. The thing is, I wrote down after the event what I thought the Pop-up window had said. As you know they vanish once clicked on.

The response to clicking on that window was "Catalog Upgrade in progress" So I'll call it "Upgrade" in future.  

[timballic]

It seems I can now backup fine with the updated Catalog, but there is now a hole for most of 2020, when the older version was not backing up as it should.

[timballic]

I wish I'd opened this thread the moment things went wrong,  two weeks ago.

I've remembered the order of events that I put at the start, wrongly.

After starting the  Catalog Upgrade, it DID reopen,  BUT only with only files up to 18.04.2018 ie everything from then up to 14.07.2020 missing. (This is what JimHess (above), couldn't understand)

I THEN reloaded from my last backup (the file labelled 18.04.2018), which again only loaded up to 18.04.2018

Now, I'm wondering, is it possible that that last backup might have created the files between 18.04.2018 - 14.07.2020, BUT put them somewhere else, or even created a second catalog, if the original was too big?

If so, where should I search for it/them??

[dj_paige · Accepted solution]

"Now, I'm wondering, is it possible that that last backup might have created the files between 18.04.2018 - 14.07.2020"

There are no partial Lightroom Classic catalog backups. It backs up everything in the catalog.

What you need to do is search all of your hard disk(s) using your operating system's search feature for all files that contain .LRCAT in the file name. On Windows, you can search for *.LRCAT (Probably on a Mac it is similar, but I don't really know). Then, when you find them, you open them one-by-one until you find the catalog with the desired photos. You might also have to search through all backups on all disk(s), in which case you should search for *.LRCAT.ZIP.

[timballic]

Thanks. I'll get started!

[timballic]

dj_paige, THANK YOU so much!

Yes Mac has a very good "search". Lots of .LRCAT and .LRCAT.ZIP files appeared in search.

I thought I'd start again, right back at the copy I'd made just before updating the catalog backup, on an external HD. I'm sure it's the very one I tried as soon as LrC reopened after that update, with 2+ years files missing. Anyway, now I have all the files back on LrC. Phew! It's been a hairy time!

Now, I want to be certain I don't lose them again!

First, I've turned on "Automatically write changes into XMP" that someone on Flickr suggested, as a failsafe measure.

But how do I ensure that LrC correctly backs up from now on? So far, I've only found the above file that doesn't have a lot missing from it? Could it be that my Catalog is too big for backup to cope with?

[dj_paige]

"But how do I ensure that LrC correctly backs up from now on? So far, I've only found the above file that doesn't have a lot missing from it? Could it be that my Catalog is too big for backup to cope with?"

Catalog size is not the issue here. You make a backup of the catalog that has all the photos in it.

[Just Shoot Me]

First there was no Catalog upgrade included in LrC 9.3 if you had already installed one of the version 9 releases, 9.0, 9.1, 9.2, 9.2.1. there was a catalog upgrade when coing from either version 7 to 8 or from 8 to 9.

So I have no idea what version you had been using before you did the LrC verion 9.3 install.

Also LrC can randomly select the wrong catalog to upgrade if you have more than one catalog file in the standard location.

I think this is what happened. LrC 9.3 chose the wrong catalog to upgrade and it was from the 4th month of 2018. So as dj_paige says you need to search your drive for another, newer catalog. All catalogs end with lrcat (LRCAT). so search your drive for files that end with lrcat.

[timballic]

Many thanks Just Shoot Me, that sounds very logical, as I certainly hadn't upgraded it for some time, (probably from ver. 7-8)

I have now found all the missing photos after doing the .lrcat search. 

[timballic]

...everything looked rosy, but now I'm finding that about a quarter to a third of all the photos since the catalog update, have either a symbol, top right, of a box containing three vertical lines and a "!" which when I hover over it says, "Metadata has conflict".

Maybe it's because I checked "Automatically write changes into XMP" when I restarted?

How do I sort this?

[timballic]

..also in the metadata Status, "Conflict detected" has appeared?

I've only edited these photos in Lightroom, and before the catalog update, there were no "warning signs"

[timballic]

Ha! I clicked one reply for "Correct answer" as requested, and every post now has "Correct answer" next to it, even my follow-on questions!