Lets talk about Photoshop Elements full backup vs. incremental backup

[BILL A. BRADY]

Lets try it this way. A have a folder that has my last catalog backup. Lets call that folder Last Complete Backup. Lets note that it has 331,000 things in it, maybe those things are files, I don't know, but 331,00 somethings.  Moving on, today I did a partial backup. during that process the software prompts, Where do you want to the new backup to go to, the Adobe noun for that place seems to be backup path. I had already made a folder. Lets say the name of that folder is the New Place, it's empty right now.  The next thing I have to provide is the path to where the current backup data is, and that's of course is Last Complete Backup folder. I provide that information. The backup runs and finishes. Now I have two folders. New place has 16,000 things in it. Last Complete Backup has 331,000 things in it.   So I'm thinking, that new folder isn't a new bakup it's a just a partial backup.  Well, why am I complaining, didn't I want a partial backup? Yes, I wanted New Place to have all the old info and the new info. I wanted Last Complete Backup to stay unchanged.  I don't think I got that because no way does 16,000 somethings have the new stuff and the old 331,000 somethings.  What i think  is I now have are two folders and between the two of them I have one complete meta data backup,  And  yes, there are now two places,but   every time I do an incremental backup   I will add another place. I'm going to anticipate a question. Hey, there, why didn't you just write over the orginal, send the new stuff to the old place? I was kind of hoping to leave the Last Complete Backup alone, just in case there's an obvious problem. Of course the problem is not obvious, I won't know until its too late  unless I want to do a fire drill after each backup.  So, to put my question into one sentence. If I create a partial backup to a new location, is my Adobe backup now is in two pieces?   

[MichelBParis]

Yes, we can talk about it, but we should not forget that there must be some reason this 20 years old incremental backup is practically never used.

The short answer to your question is that your partial backup will need its own backup folder. If you decide to create a full backup and then incremental monthly backups, you'll get 13 folders. Needless to say, you'll have to name those 13 folders in a descriptive way to follow the required restoring step. Then,

- you'll be able to restore your library and catalog to any of the monthly stages.

- Those 13 folders will take much less space than 12 full backups

- You'll have to follow blindly the restoring instructions, not the workflow you imagine would be good or better.

- You won't know if the backups are valid until your try a restore. Most users try it too late and forget about incremental backups...

- Incremental backups and restores won't save as much time as you expect.

That's the short answer. Since there is no Adobe description of the internal workflow, we can discuss what we imagine is the real process, but that may take some time!

[BILL A. BRADY]

Thank you, Mr. Paris, for your reply. 

I have written your answer  on paper so I can parse it as if it were a opinion written by the United States Supreme Court. I'm not being at all sarcastic, I see a lot to think about here before I ask a few more questions later if I may. 

 The backup that prompted me to ask for community help was performed before installing Elements 2022.  I had been using 15.  I've included a screen shot of the Elements 2022  backup screen; there  are now three choices. Previously, version 15 only had two. So, I have a more basic question now. I believed that Elements 15, and earlier versions, didn't back up photos, these earlier versions backed up the catalog meta data, For me, that meant the Tags. For  the Elements software, I'm only guessing: the backup entailed whatever Elements  needed to know to track where the photos were. With Elements 2022, Options two and three seem to be suggesting that I can back up photos (and videos) inside the software.  Is that true? 

I'm easily impressed with myself for just finding this post again. I have also stumbled upon what seems to be the equivalent of a users manual here. Hoping I can find both this post and the manual again. If so,I will take some time to read through the manual. That should improve the quality of my side of the conversation. Again, thank you for your reply. 

 

[MichelBParis]

Hi Bill,

You are right that Elements 2021 and 2022 have a new option, but it's the contrary of what you seem to think. From the beginning, nearly 20 years ago, the backup process of the Organizer did save both the catalog (the tags and organization data) AND all your photo and video files. The restore process from a catalog full backup can restore everything on a new computer or drive while keeping the links to the new physical location of the photo files.

Other management systems, like Lightroom, only care for backups of the catalog (the database, tags and hierarchies) and don't backup at all the photo and video files, leaving the users (supposed to be more 'professional') to choose his/her favorite backup software. Of course, backing up a catalog folder is far faster than backing up all your photo/video files. So, what is new in the organizer is the option to work like in Lightroom, with a backup of only the catalog folder. You are free to use your preferred backup or syncing system for the photo files. That's obviously much faster, since you can always spend a couple of minutes to backup the catalog after each editing session instead of running a full backup for a full night job in a very big photo library. You'll make much more frequent catalog backups which will protect you much better from user errors or occasional catalog issues. What is not discussed in detail in the help docs is the limitations of catalog only backups vs catalog+library backups.

Suppose you have a good backup or syncing software to take care of your photo files. You lose your photo files (1 = accidental deletion) or (2 - theft, fire, flood...). In both cases, you can recover your photo files. In situation 1 the new catalog backup will take care of the catalog and all will be well. In situation 2 you'll recover your photo files on a NEW and different drive, and the backup of the catalog will NOT find the files in the original location. All your files will be seen as missing/disconnected. Lightroom can reconnect to the new location but the organizer will not offer a practical reconnection for a big library. This means that you are not fully protected as if you had done a full organizer backup.

The logical safe workflow if you want to use the new catalog only backup is to make also periodical full backups in addition to the fast and frequent catalog only backups. Let's say a full backup every month after daily catalog backups. If you have a serious issue necessitating a restore on a new drive, you recover everything except the last month. The most recent files can often be retrieved partly, so the damage may not be too important.

If you choose an external backup or syncing solution, you get the advantage of more speed and automated tasks.

You won't lose anything if you can restore on the original computer but if you have to restore to another computer/drive, the links in the catalog will be disconnected. If you have also a recent full backup, your situation will be manageable: the new files and edits after the full backup are recovered, the daily catalog backups recover your editing work since the full backup, and you only have to reconnect a manageable amount of files.

To simplify, users with small catalogs and not too 'techy' should use regular full backups and I advise them to also use the catalog only backups.

Advanced users with big catalogs

- should find the best external backup or syncing solution to speed up the backups and make them frequent enough

- make very frequent catalog only backups (from the organizer automatically or from their own solution, a simple catalog copy is efficient enough)

- and ALSO make organizer full backups periodically (monthly, every 3 months...)

[Hcook43]

I still don't understand what the incremental backup backs up. I thought most back up systems goes to the previous backup and leaves  data unchanged that hasn't changed and changes things that have changed giving a new backup that matches the current source folder. If I start to do an incremental backup on PSE 2019 to the same folder that holds the previous backup. i get a message saying a backup already exists there, do I want to overwrite it. Again I don't know what that means. What happens it I say yes? Will I have a folder containg only the data that has changed since the previous backup? Obviously I am confused.

[MichelBParis]

Hi @Hcook43 ,

Indeed, the online help does not provide a detailed answer about incremental backups.

https://helpx.adobe.com/elements-organizer/using/backup-restore-catalogs.html

I have never found any tech doc about how that works internally, but the management of the backups works like this.

The first thing is to understand what a normal FULL backup does. It creates a backup folder. not a 'packaged' container file. The name of the backup folder is for you to choose, and it's important to choose a significant name, like BK20220727Full. That folder contains among others:

- a renamed copy of the main sqlite database file in the catalog folder: catalog.pse20db (20 is for version 20). It is always renamed catalog.buc.

- renamed copies of all your media files and other catalog components items

- a renamed copy of your thumbnail cache

- and  other items covering all the items in the catalog folder.

- most important: a special file, backup.tly which contains all the info about the original media files structure and paths to be able to restore to a new destination.

So, trying to save an incremental back INTO an existing backup folder will completely destroy that existing folder. Any incremental backup will work just like a full backup with the same backup folder structure, the same components.

So, you start with a full backup and you create as many incremental backup you may need in their distinct folders.

To restore, you normally choose the last incremental backup folder, and the program asks you to open the original full backup folder and/or other previous incremental backups.

That explains why too many users are not able to restore successfully.

What is good to know is that in each incremental backup folder you will only find copies of the new or modified files, which is expected, but also the 'catalog .buc' will be a simple renamed copy of the catalog.pse20db in the state of the incremental backup date. It includes the info about all files at that date. The backup process is able to compare the 'catalog.buc's of the original full and the incremental backups to restore new and modified files while skipping the deleted files. That why the process needs to compare the backup versions in their own distinct folders.