Corrupt Catalogue file - about 5 years of work lost.

[Rick Gerard]

I'm syncing my Lightroom Clasic through the Creative Cloud files. Satuday I uploaded 2 new 128 GB cards and started developing the images. When i quit lightroom it hung up, would not close and the application became unresponsive. I'm running a fully loaded iMac Pro, OSX 10.14.5 and Lightroom is up to date as of today. All original files are stored on an mirrored external thunderbolt drive with 5 enterprise level drives. The gear is all first rate.

On Sunday evening I opened up Lightroom to keep editing and I got the dreaded catalogue file corrupted error. After the failed repair and shutdown I copied the entire folder, moved it to a new location for safety, then tried opening Lightroom again with no luck. An internet search revealed a way to repair the LRCAT file using the terminal and some SQL tools built into the Mac. I did this and the first time everythig looked like it worked but the file faled to open again and tried to go through the repair. I figured I would open up one of the zip files from the backups that I do twice a week but sadly, those files would not open either. There also does not seem to be any way to get Lightroom to open up and ask you for a catalogue so the constqnt crashes were not giving me a chance to try something else and if I just tried to open Lightroom by opening a backup file it would crash again and write a bunch of new data to the existing files.

I even went back to a Time Machine backup and triedd to open that. Now I have about 7 copies of my LRCAT that are reporting as corrupted and can't be fixed by any method I have tried. I have even uninstalled Lightroom, rebooted the computer, deleted a bunch of the files from the Library/Applicaiton Support/Adobe folder which was a suggestion from this forum. I finally gave up and just created an entirely new catalogue by renaming the LRCAT files, moving them, and starting a new Catalogue in a new folder. Almsot 4 hours later my images were all back in place, the folders were all where they should be but the collections were all gone and I think that the adjustments to many of the images are also fouled up. I have not had a chance to verify that yet.

So here's my quesiton, has anyone had this kind of caterstrphic failure happen before? Do you know any other solutions to repair the LRCAT files?

It also turns out that my Creative Cloud folder isn't updating. It's been syncing for more than 4 hours and it's is still showing 4 out of 5 and 2 seconds remaining even after three re-boots of both systems.

This is a real mess. About 28,000 images, hundreds of catalogues and months of work apparently gone. At least all of the images are backed up. I really want to know why the backups won't open - not one of them, and I want to know why the Lighroom won't give you the opportunity to choose a library when you open it. There shold also be an option to save your library as a different name like you can in every other app so you can generate backups and copies whenever you want.

Any ideas on how to recover the corrupted catalogue files wuld be appreiated. I think that I'm going to have to go back to a system where each project has it's own LRCAT file because I can't afford to work for another year and then loose all of my sorting and organizing.

[Sahil.Chawla]

Hi Rick,

Sorry to hear about the corrupted catalog, I've sent you a PM, please check & respond.

Regards,
Sahil

[dj_paige]

What happens if you take one of your backups and put it on a different disk than the original was stored on? Will it open from this different disk?

[JohanElzenga]

How did you try to open these catalog backups? I have often seen that people try to open a file directly from the backup disk, but that is not the way to do it. You have to use Time Machine to restore the file(s) to another disk.

[dj_paige]

Also, you probably did this, but I'll mention it anyway ... backup catalogs have to be unzipped first.

[Bob Somrak]

https://forums.adobe.com/people/Rick+Gerard  wrote

and I want to know why the Lightroom won't give you the opportunity to choose a library when you open it.

You can have Lightroom ask which catalog (not library) you want to open by holding the OPTION key down when you start Lightroom.

[Rick Gerard]

I unzipped the backups. None of them going back 3 months would open. I tried 2, then uninstalled Lightroom removing preferences, rebooted and reinstalled, then I tried 4 more and none of them would open on my laptop. I went through preferences and enabled lightroom sync and now, even the new catalog that will open on my iMac Pro has problems on the laptop. It shows no images.

I've got more work to do in diagnosing this problem. There is absolutely no excuse for Lightroom to be so fragile. I cannot open any backup files on either machine. I cannot copy the entire catalog file to an external drive and have it open on my iMac. I cannot connect my storage drive array to the laptop and have it open the catalog without saying the catalog is damaged. Every time Lightroom tries to repair a damaged catalog it is permanently damaged.

This is a real mess.

[dj_paige]

What happens if you put the backup on an external disk and open it from there?

[Rick Gerard]

The original files, every one of them that I try, will show up as damaged and be destroyed if I try and open them on my laptop. I even tried creating a new catalog on the laptop and opening it. It is reporting as damaged and then when I try and open the repaired file, Lightroom crashes immediately.

Nothing changed on the MacBook Pro. No or app updates. Nothing in a virus scan.

The only thing I can think of is that syncing lightroom through Creative Cloud is somehow fouling things up. Now I can't even get File Sync to turn on.

[Rick Gerard]

Syncing has started working again but it's incrdibly slow. I'm on fiber at about 90 up and down:

Syncing has been going on for at least 5 hours undisturbed. The entire Creative Cloud folder is 42GB and I'm being told that sync is going to take another 36 hours. My Lightroom folders are 9 GB. Something is wrong here.

[dj_paige]

https://forums.adobe.com/people/Rick+Gerard  wrote

The original files, every one of them that I try, will show up as damaged and be destroyed if I try and open them on my laptop. I even tried creating a new catalog on the laptop and opening it. It is reporting as damaged and then when I try and open the repaired file, Lightroom crashes immediately.

This is not what I suggested. Do not copy the backups to the internal drive of your laptop. Copy the backups on a different disk, like an external HD, and see if they will open from there.

[Rick Gerard]

dj_paige, It doesn't matter where the backup copies are. It doesn't matter which copy of a backup catalog I open, they all report as damaged and after they are repaired Lightroom crashes immediately on open. I even tried turning off Creative Cloud sync, archiving files on both machines, repairing my creative cloud installation, then verifying the Creative Cloud files stored on the web, deleting almost all of them leaving only the Mobile files, then I started enabled Sync on my iMac pro, put a copy of the new working Lightroom Catalogue in the new Creative Cloud folder so it was the only option, verified that it worked on the iMac pro, then enabled sync on the laptop which had no Creative Cloud Files folder. A new folder was created, the files started to sync and I get this progress report at the bottom of the CC app:

130,848 files, at least I think they are files and 40 hours to sync a 9 GB file that finder reports as having only 168 items. Something is all fouled up. I'll wait a while, check the sync, then try again. I'll also try to open a copy of the new library file that is working on the iMac Pro from an external drive.

Whatever is happening, I am convinced that the LRCAT files are way too fragile and the repair process is killing all of my recent files. I even sent a damaged file to Adobe to see if they could repair it. I have not heard back yet, but this is a mess. that has put me 2 days behind on the delivery of a project and eaten up about 25 hours of billable time and at least 10 hours of family time that is priceless.

[Todd Shaner]

https://forums.adobe.com/people/Rick+Gerard  wrote

I'm syncing my Lightroom Clasic through the Creative Cloud files. Satuday I uploaded 2 new 128 GB cards and started developing the images. When i quit lightroom it hung up, would not close and the application became unresponsive. I'm running a fully loaded iMac Pro, OSX 10.14.5 and Lightroom is up to date as of today. All original files are stored on an mirrored external thunderbolt drive with 5 enterprise level drives. The gear is all first rate.

Why are you storing and syncing the original image files and LR Catalog files in the Creative Cloud Folder and when did you start doing this? LR Classic users normally only sync LR Collections, which uploads much smaller smart previews to the Cloud. It sounds like you're confusing LR Classic's operation with LR Desktop, Mobile, and Web apps. I suggest you Pause Creative Cloud File Sync for now until you resolve the corrupt catalog issue and stop using the designated Creative Cloud Folder for your original image files and LR Catalog files storage location. This is most likely what's causing the LR hangs and catalog corruption messages. Do you also have the Lightroom Desktop 2.3 app installed or just the Classic 8.3.1 app? Go to LR Help> About to see the version number.

[Rick Gerard]

I'm not storing original image files in the Creative Cloud folder. Only Lightroom Library Files right now. I deleted everything else.

I paused sync on both machines about 3 hours ago. Then I put 3 folders in the Creative Cloud Folder on my iMac Pro. The Test Cat folder contains a catalog with 10 images I created yesterday as a test. The Lightroom folder contains a copy of my original catalog from a 3-month-old backup that I have not tried to open in either computer yet because every catalog that I have opened so far has caused Lightroom to crash and corrupt the file. The Master copy folder is a new catalog that I created on the iMac pro from my external drive that stores all of my images. There are no collections and I am not sure that all of the photo edits have been preserved.  It took the web about 3 minutes to sync up.

Then I turned on syncing on my MacBook Pro and after an hour here's what I have:

That's right, 161 hours to sync 3 folders. Something is fouled up.

I loaded the same 3 folders in DropBox to see how long it took for them to sync up. Under 3 minutes and they were on the MacBook Pro.

Something is definitely fouled up with syncing and I think that it has something to do with the problem Lightroom is having opening up my previous files.

Here's what happens when I try and open up the catalog file in Dropbox:

Same old error. If I repair the file that fails, then I can't open it up on the iMac Pro either. Using terminal and the SQL fix won't fix the file either.

Something on my MacBook Pro is killing catalog files and fouling up sync. I wish I knew what it was.

[Todd Shaner]

In your reply I don't see that your tried what I suggested.

"I suggest you Pause Creative Cloud File Sync for now until you resolve the corrupt catalog issue and stop using the designated Creative Cloud Folder for your original image files and LR Catalog files storage location."

For troubleshooting purposes let's keep it simple without ANY syncing and see what we can do. Place one of the unzipped Lightroom Catalog.lrcat backup files and nothing else in any folder other than the Creative Cloud Folder and pause syncing. Double-click on the .lrcat file in that folder and it will launch LR and should open. Please tell us what happens. Thank you!

[Rick Gerard]

I've done that several times. Every time on my MacBook Pro, I get the warning I showed in my previous post with this screenshot:

Then if I let Lightroom try and repair the file Lightroom immediately crashes right after it opens. I have uninstalled Lightroom using the CC app and reinstalled it using the CC app. I have uninstalled using the uninstaller in the Applications folder, I have purged everything I can think of purging and nothing works. No existing catalog can be opened without failing and every time a repair is attempted by Lightroom the file is permanently damaged.

[Per Berntsen]

It seems that you haven't tried to reset the preferences.

Probably won't help, but it's worth a try,

https://www.lightroomqueen.com/how-do-i-reset-lightrooms-preferences/

[Rick Gerard]

Did that too. I don't think I have missed anything, but I just may have found something. I made a copy of just the LRCAT file and put it in another folder so there were no preview or helper files. Lightroom opened just fine. No Previews because the original files were not there. When I closed the all 3 files were in the folder. I could open the catalog again.

Later tonight I'll attach the drive that contains the originals and see if previews can be built and saved, edit a couple of photos, then see if I can close, unplug the originals, and open LR again. If that works, and I can open the file on the iMac Pro and the MacBook Pro, maybe there is a chance to save one of the backups of the original catalog so I don't lose 5 years of creating collections and making notes.

[JohanElzenga]

https://forums.adobe.com/people/Rick+Gerard  wrote

Did that too. I don't think I have missed anything, but I just may have found something. I made a copy of just the LRCAT file and put it in another folder so there were no preview or helper files. Lightroom opened just fine. No Previews because the original files were not there. When I closed the all 3 files were in the folder. I could open the catalog again.

Later tonight I'll attach the drive that contains the originals and see if previews can be built and saved, edit a couple of photos, then see if I can close, unplug the originals, and open LR again. If that works, and I can open the file on the iMac Pro and the MacBook Pro, maybe there is a chance to save one of the backups of the original catalog so I don't lose 5 years of creating collections and making notes.

It is indeed possible that this error is generated because of the previews file. It could be corrupt. Or did you perhaps try to exclude the previews from the Creative Cloud Files folder by placing them elsewhere and placing an alias to them in the catalog folder? So the large previews file would not be synced? That will also generate this error. You cannot fool Lightroom with an alias, only with a symbolic link.

[Todd Shaner]

https://forums.adobe.com/people/Rick+Gerard  wrote

I just may have found something. I made a copy of just the LRCAT file and put it in another folder so there were no preview or helper files. Lightroom opened just fine. No Previews because the original files were not there. When I closed the all 3 files were in the folder. I could open the catalog again.

Good to hear following my suggestion the LR catalog file opens with no issues. So now let's investigate exactly WHAT and WHY you are trying to sync using the Lightroom Classic Creative Cloud Files folder. Please be very specific!

[Rick Gerard]

If you sync the files then you can edit anywhere using the previews. I take the laptop in the field load and catalog files, do some initial editing all with the files on an external drive. When I get back I plug the external drive into my desktop and, backup to the archives and then just re-point the folder or if there are just a few images I just drag the folder with the most recent footage to the storage array right in the Library in Lightroom. It's a huge time saver.

Another workflow happens when I'm in the field and a client requests an update on an existing image. I can send him low rez proofs from my laptop, then, even if I have to use cellular data, I can remotely connect to the storage drive in the office through my network, download a copy of the original to the laptop, and finalize the product in the field.

If I need to use Lightroom Mobile for anything I can also sync through the Creative Cloud storage.

This is standard practice for a lot of professionals. I have access to the last 5 years work anywhere in the world and can edit, make collections, and prepare proofs without an internet connection. The new collections and edits will update as soon as I connect to WiFi.

The problem did not turn out to be syncing through the Creative Cloud Folder, it turned out to be corrupted preview files.

There are a bunch of tutorials, some of them excellent, on the workflow.

All of my professional work is in one Catalog, all of my experimental in another, all of my family in another. The catalog that was ruined was the most recent professional catalog, but at least now I know how to restore it across my machines from a backup that is only about a month old.

I have resorted to running a backup every time I edit in Lightroom, and I've filed away the knowledge that rebuilding a catalog with Lightroom may be unsuccessful and kill the catalog forever and rebuilding the previews folder may fix the problem.

[Todd Shaner]

https://forums.adobe.com/people/Rick+Gerard  wrote

The problem did not turn out to be syncing through the Creative Cloud Folder, it turned out to be corrupted preview files.

You do realize this workflow is undocumented by Adobe and therefore unsupported.

How to sync Lightroom Classic photos with Lightroom

I'm not saying you shouldn't use it, but Caveat Emptor. My guess is that during one of your editing sessions some Library Preview updates were still pending and not properly synced in the previews.db database file. ERGO-Previews corruption. Normally when the Previews database becomes corrupted you get an error message and can simply delete he Previews folder and rebuild them. Obviously with the LR catalog located and synced in the Creative Cloud Folder the behavior is different.

To prevent this from happening in the future select ALL of the files after each editing session or import and go to menu Library> Previews> Build Standard-Sized Previews. Wait for the progress taskbar to complete. Also make sure the Cloud Sync indicator is not active. Then and only then close LR and shutdown your system. If you're doing this already it's possible there is another "undocumented" issue that will cause it to happen again.

Also as JohanElzenga mentions don't try to exclude the Previews folder from the Creative Cloud Files folder by placing them elsewhere and placing an alias to them in the catalog folder. Even though the Previews aren't needed for your workflow you still need to sync them along with the Smart Previews.

This is an unsupported workflow that Adobe could easily implement to work better. However, Adobe has stated emphatically that "No new Cloud Sync features will be added to Lightroom Classic."

[Rick Gerard]

Syncing through Creative Cloud is a feature outlined right in the help files. Right there in your link, and here, complete with video tutorials,

How to sync photos and edits across all devices | Adobe Photoshop ...https://helpx.adobe.com › Adobe...

And there are a lot of other articles on the capability.

Skip Lightroom CC and Sync Catalogs With LR Classic | Fstoppershttps://fstoppers.com › Education

I am not doing anything at all outside of the recommended procedures. Maybe I didn't explain the workflow carefully enough, but I did not customize anything. It was Lightroom trying to repair a catalog that caused all the problems. The huge amount of work that was lost was the collections that were synced.

There is also no reason at all that letting Lightroom rebuild a catalog should completely destroy it. That is what is happening. I found a couple of techniques to read and analyze the damaged LRCAT files using Terminal. Before the Lightroom attempts to repair a catalog the SQL tools return no errors. The technique does not fix the damaged copy, but no errors are returned in the hundreds of lines of code that are echoed in the Terminal panel.

After Lightroom tries to repair the file there are hundreds of lines of error messages. Something is broken that should not be broken.

Turns out the only way to fix the problem is to isolate the LRCAT file in another folder, make sure the original files are available, then force Lightroom to rebuild the previews and start all over again. If you use a separate LRCAT for every job this is not that much of a big deal, but if you keep a couple of terabytes of files or even a couple of months of work in a single catalog you could lose everything if you let Lightroom try and rebuild the catalog.

[Rick Gerard · Accepted solution]

I'm calling this the correct answer.

If Lightroom locks up and tells you that the catalog is damaged DO NOT let Lightroom try and repair the catalog. Force Lightroom To Quit without running the repair, it's not easy, then try the following.

1. Reboot
2. Copy the LRCAT file to another location but do not move any of the other files
3. Make sure that the original image files are available and in their normal location
4. Hold down the Alt/Option key when opening Lightroom so you can choose the LRCAT file copy or double click the copy of the LRCAT file to open Lightroom
5. Recreate all image previews in the Lightroom Library using the recommended technique
6. Verify that all collections and synced files are showing up as they should.
7. Open View On Web to see the Creative Cloud Folder in a browser
8. Pause Creative Cloud File Sync
9. Delete the broken file using the web browser
10. Replace the broken catalog files in the Creative Cloud Folder using all of the repaired files in the temporary folder
11. Turn Sync back on and wait for all files to be updated
12. Run Lightroom again.

I tried to have Lightroom repair a broken catalog 5 or 6 times using different backups. Every single time the files were not repaired and Lightroom would crash immediately when it tried to open the supposedly repaired file again.

[dj_paige]

Copy the LRCAT file to another location but do not move any of the other files.

I was trying to tell you this.