Inspiring

Answered

Incremental backup files not created in backup folder

Forum|Forum|3 years ago
September 30, 2022
2 replies
594 views

I backed up my PSE 2021 catalog to a folder on a drive. When I do an incremental backup, I select the correct folder but the new files are all dumped into the root directory. How do I get the software to drop these folders into the indicated folder? The Backup.tly file is located in the correct folder, BTW.

Windows 10, Dell XPS, 32GB memory, i9 processor, 10 core.

Thanks.

This topic has been closed for replies.

Correct answer MichelBParis

I backed up my PSE 2021 catalog to a folder on a drive. When I do an incremental backup, I select the correct folder but the new files are all dumped into the root directory. How do I get the software to drop these folders into the indicated folder? The Backup.tly file is located in the correct folder, BTW.

Windows 10, Dell XPS, 32GB memory, i9 processor, 10 core.

Thanks.

By @WSC33

Each backup IS a folder, whether it's a full or an incremental one.

You always have to specify a dedicated folder each time you create a backup. Either you specify that folder in the backup starting window to create a new one, or you create a dedicated one before starting the backup process and guide the backup creation to that newly created empty folder. So, the backup process requires that YOU define the backup folder and its folder name as well as the location (containing folder) where you'll store the backup folder. For an incremental backup, you'll have a backup folder for the full backup, and a new one for each incremantal backups. Those backup folders can be stored directly under the root folder or under a common containing folder with the full and incremental backup folders of a catalog.

NEVER specify an existing backup folder as the containing folder! Take care to name each created backup folder with a descriptive name (for instance the catalog name and a date).

If normal contents of a backup (ie catalog.buc, backup.tly, b00... files) appear directly in the ROOT folder, that would mean that you have not specified the correct location at the start of the process.

Warning;

Incremental backups may be dangerous if you don't fully understand how they do work and if you don't follow exactly the prompts in the restore process.

NEVER rely on incremental backups before trying successfully a complete backup and restore process. Until the result is not what you expect, keep using normal full backups. Incremental backups don't save you a lot of time.

MichelBParisCorrect answer

Adobe Expert

I backed up my PSE 2021 catalog to a folder on a drive. When I do an incremental backup, I select the correct folder but the new files are all dumped into the root directory. How do I get the software to drop these folders into the indicated folder? The Backup.tly file is located in the correct folder, BTW.

Windows 10, Dell XPS, 32GB memory, i9 processor, 10 core.

Thanks.

By @WSC33

Each backup IS a folder, whether it's a full or an incremental one.

You always have to specify a dedicated folder each time you create a backup. Either you specify that folder in the backup starting window to create a new one, or you create a dedicated one before starting the backup process and guide the backup creation to that newly created empty folder. So, the backup process requires that YOU define the backup folder and its folder name as well as the location (containing folder) where you'll store the backup folder. For an incremental backup, you'll have a backup folder for the full backup, and a new one for each incremantal backups. Those backup folders can be stored directly under the root folder or under a common containing folder with the full and incremental backup folders of a catalog.

NEVER specify an existing backup folder as the containing folder! Take care to name each created backup folder with a descriptive name (for instance the catalog name and a date).

If normal contents of a backup (ie catalog.buc, backup.tly, b00... files) appear directly in the ROOT folder, that would mean that you have not specified the correct location at the start of the process.

Warning;

Incremental backups may be dangerous if you don't fully understand how they do work and if you don't follow exactly the prompts in the restore process.

NEVER rely on incremental backups before trying successfully a complete backup and restore process. Until the result is not what you expect, keep using normal full backups. Incremental backups don't save you a lot of time.

W

WSC33Author

Inspiring

Wow. Thanks for the detailed and helpful answer. This is positively frightening and unlike any other backup software I’ve ever encountered. And certainly not intuitive at all from PSE. I’ll attach a screenshot that I see for my incremental backup. There’s little indication of the information you have provided, especially the necessity for a dedicated folder for incrementals.

I'm now assuming from the screenshot that backup path means that I should have added a new folder name (Backup 1, e.g.). How would I know that? I assumed that it went into the original backup folder where the “Backup.tly” file is located. And to make it even better, I moved the incremental files into the main backup folder, apparently a big “no-no.”

I can’t remember exactly, but my original full backup took many hours. I have about 36k pics.

So, in summary, is it fair to say that it’s really best to delete your old backup and do a whole new one each time or keep multiple full-back ups. I had no idea this was so complicated.

I backup my photos (most of which have the metadata embedded) with other software, so my collection is covered. The restore test sounds like a lot of work. I'm assuming I can restore to a different drive? I don’t have to put it back in the same place it came from, which would require me to delete all of my pics to test. Not an attractive thought.

Did I get this right?

Thanks again!

MichelBParis

Adobe Expert

Wow. Thanks for the detailed and helpful answer. This is positively frightening and unlike any other backup software I’ve ever encountered. And certainly not intuitive at all from PSE. I’ll attach a screenshot that I see for my incremental backup. There’s little indication of the information you have provided, especially the necessity for a dedicated folder for incrementals.

I'm now assuming from the screenshot that backup path means that I should have added a new folder name (Backup 1, e.g.). How would I know that? I assumed that it went into the original backup folder where the “Backup.tly” file is located. And to make it even better, I moved the incremental files into the main backup folder, apparently a big “no-no.”

I can’t remember exactly, but my original full backup took many hours. I have about 36k pics.

So, in summary, is it fair to say that it’s really best to delete your old backup and do a whole new one each time or keep multiple full-back ups. I had no idea this was so complicated.

I backup my photos (most of which have the metadata embedded) with other software, so my collection is covered. The restore test sounds like a lot of work. I'm assuming I can restore to a different drive? I don’t have to put it back in the same place it came from, which would require me to delete all of my pics to test. Not an attractive thought.

Did I get this right?

Thanks again!

By @WSC33

Hi @WSC33

You are absolutely right about:

- the backup and restore process, especially the incremental backup, being non intuitive and even far from any other backup or syncing solutions available totay. The help docs are no better.

- What is unique with the organizer backup solution is that (a) it backs up both the media files and the catalog (b) the catalog it backs up is a also a folder.

'a) catalog folders vs backup folders? catalog folders get the name given to the catalog in the organizer. They contain the main sqlite database (catalog.psedb21 for PSE 2023), the thumbnail cache, the information about face recognition, visual similarity and many other data about the status of the catalog.

On the other hand, backups are also catalogs. They contain renamed copies of all media files (B0...), a file 'catalog.buc' which is a renamed duplicate of the main sqlite database, ei catalog.psedb21 for PSE 2023, a necessary backup.tly file to keep the links of each media file to the folder structure in your OS browser, a copy of your thumbnail cache and renamed versions of all other resources necessary to store the information about face recognition, visual similarity etc.

If you understand the main components of a catalog folder (with its main database, thumbnail cache and various components), you'll see that including the contents of a different catalog folder into the same folder of a different catalog will break the relations between the different components so that the hybrid catalog can't work. If the normal catalog folders had been 'zipped' into a single file instead of staying a common open folder, there would have been no risk of mixing two catalogs by error. Note that Lightroom (a bit younger than the organizer) has a single database file instead of a folder and a separate thumbnail cache. And Lightroom does NOT backup your media files, only the catalogs.

Now, what about incremental backups?

As already stated, each one is created as a standard catalog structure. It can't be included it in any other backup catalog folder. The expected interest of an incremental backup is to store anything which is new or missing compared to the previous backup version. Generally, only a small part of the backup media files has been changed. So the backup process has to find a way to compare the present situation to the previous one to store only the changed or new files in the incremental backup folder. And the process and algorithm of that comparison is described nowhere. It's also unique to the organizer backup process. I have my own idea about how that is performed, but to keep it relatively short, the fact is that in each incremental backup folder, you get a 'catalog.buc' which is a simple renamed version of the current main database. It's much faster to compare that catalog.buc to the one of the previous backup to determine which media files have been added or changed in between and to store only them in the current incremental backup. The restore starts with the chosen incremental version catalog.buc (easy to rename back to the original catalog.pseXXdb). The process is much faster and the resulting folder only contains what has been changed. The practical advantage is that if you have 6 incremental backups, you can start with the number 4 one for instance and restore everything at any of the previous incremental versions status. The restore process first recreates the full catalog from the renamed catalog items. Then, it will prompt you to open the previous versions whan needed. First the media items in the last version will be restored, then media items present in the previous incremental backup not already restored and so on untl the original full backup. Other advantage, each backup step only has to store changed and added items, no worry about deleted files since you start with the most recent catalog.buc.

That was to explain why it's important to follow precisely the prompts of the restore process to open the right incremental backup folders in the required order. And the importance of naming the incremental backup folders clearly...

W

WSC33Author

Inspiring

Anybody have any ideas here?

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded