Highlighted

Photoshop Elements "Backup Catalog" questions

Explorer ,
Jun 10, 2020

Copy link to clipboard

Copied

First, am I correct in understanding that "Backup Catalog" makes a backup of not only the Elements' "Catalog" but also the entire photo library that the Catalog points to?

 

When you select "Backup Catalog" in Photoshop Elements 2020 (Windows), then select Full Backup, the resulting files and folders bear no resemblance to the original. It looks totally proprietary. Why?

 

For backing up one's photo library, is it okay to use a separate backup program (e.g., Syncback), provided it retains the same folder structure as the original and retains all attributes and timestamps? Likewise, how about simply making a copy of the catalog in C:\ProgramData\Adobe\Elements Organizer\Catalogs?

 

I prefer to avoid proprietary backup schemes because the results are inaccessible and unusable if corrupted in the slightest way. 

----

Update: Case closed.  Thanks to the kind folks here I was able to accomplish my goal of putting the photos and the catalog on an external USB drive. I used PE2020's built-in backup/restore.  It was quick and easy.

Adobe Community Professional
Correct answer by MichelBParis | Adobe Community Professional

trwalp,

The answer to your question is detailed in many of the posts I did those last days.

For backing up one's photo library, is it okay to use a separate backup program (e.g., Syncback), provided it retains the same folder structure as the original and retains all attributes and timestamps? Likewise, how about simply making a copy of the catalog in C:\ProgramData\Adobe\Elements Organizer\Catalogs?

Making a copy of the catalog folder does not work if you have to restore on a different drive (or drives) as the original ones. You can copy or move the catalog folder practically everywhere, but its contents don't change. All links stored in the catalog for the location of the pictures include not only the path in the folder trees, but the identification of the drive. If the drive is missing, all files in the copied catalog point to a missing drive. The catalog is unusable. Also, the present 'reconnection' tool in the Organizer can't reconnect a big library, contrary to what Lightroom can do.

 

So, any external backup tools will work only if the restore is done on the same original drive(s). Or at least with a new drive with the same identification, which would require a cloned drive, a Windows command to change the drive identification or a change of the contents of the Organizer with an Sqlite tool.

 

I prefer to avoid proprietary backup schemes because the results are inaccessible and unusable if corrupted in the slightest way. 

Just consider zipping your media folder tree: it's a valid backup scheme even if you don't understand what's in it. The real issue with the organizer catalog in this respect is that you can look into it and alter its contents. The backup folders contains

- a 'catalog.buc' file which is a renamed copy of the 'catalog.pseXXdb' database.

- renamed copies of each of the media files (including xmp sidecar files)

- a special file 'backup.tly' which stores lists of files to restore the folder structure and manage your backups, full or incremental. If this file is missing, that means your backup is corrupt. It's the last file written into the backup folder.

- other accessory data or database such as the thumbnails cache and face recognition and other data.

 

So, using an external backup tool can't work, and that's true for all catalog based softwares. 

 

For me, the real issue with the old backup scheme in the organizer is that you can't easily 'reconnect' to the same folder structure like what is possible in Lightroom. I suppose that trying to apply the same workflow in the organizer would require a total overhaul of the catalog structure and management. However, I can imagine that with today's growing power of AI, Adobe could manage to improve its reconnection feature if the folder structure is exactly the same.

 

TOPICS
Organizer

Views

338

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more

Photoshop Elements "Backup Catalog" questions

Explorer ,
Jun 10, 2020

Copy link to clipboard

Copied

First, am I correct in understanding that "Backup Catalog" makes a backup of not only the Elements' "Catalog" but also the entire photo library that the Catalog points to?

 

When you select "Backup Catalog" in Photoshop Elements 2020 (Windows), then select Full Backup, the resulting files and folders bear no resemblance to the original. It looks totally proprietary. Why?

 

For backing up one's photo library, is it okay to use a separate backup program (e.g., Syncback), provided it retains the same folder structure as the original and retains all attributes and timestamps? Likewise, how about simply making a copy of the catalog in C:\ProgramData\Adobe\Elements Organizer\Catalogs?

 

I prefer to avoid proprietary backup schemes because the results are inaccessible and unusable if corrupted in the slightest way. 

----

Update: Case closed.  Thanks to the kind folks here I was able to accomplish my goal of putting the photos and the catalog on an external USB drive. I used PE2020's built-in backup/restore.  It was quick and easy.

Adobe Community Professional
Correct answer by MichelBParis | Adobe Community Professional

trwalp,

The answer to your question is detailed in many of the posts I did those last days.

For backing up one's photo library, is it okay to use a separate backup program (e.g., Syncback), provided it retains the same folder structure as the original and retains all attributes and timestamps? Likewise, how about simply making a copy of the catalog in C:\ProgramData\Adobe\Elements Organizer\Catalogs?

Making a copy of the catalog folder does not work if you have to restore on a different drive (or drives) as the original ones. You can copy or move the catalog folder practically everywhere, but its contents don't change. All links stored in the catalog for the location of the pictures include not only the path in the folder trees, but the identification of the drive. If the drive is missing, all files in the copied catalog point to a missing drive. The catalog is unusable. Also, the present 'reconnection' tool in the Organizer can't reconnect a big library, contrary to what Lightroom can do.

 

So, any external backup tools will work only if the restore is done on the same original drive(s). Or at least with a new drive with the same identification, which would require a cloned drive, a Windows command to change the drive identification or a change of the contents of the Organizer with an Sqlite tool.

 

I prefer to avoid proprietary backup schemes because the results are inaccessible and unusable if corrupted in the slightest way. 

Just consider zipping your media folder tree: it's a valid backup scheme even if you don't understand what's in it. The real issue with the organizer catalog in this respect is that you can look into it and alter its contents. The backup folders contains

- a 'catalog.buc' file which is a renamed copy of the 'catalog.pseXXdb' database.

- renamed copies of each of the media files (including xmp sidecar files)

- a special file 'backup.tly' which stores lists of files to restore the folder structure and manage your backups, full or incremental. If this file is missing, that means your backup is corrupt. It's the last file written into the backup folder.

- other accessory data or database such as the thumbnails cache and face recognition and other data.

 

So, using an external backup tool can't work, and that's true for all catalog based softwares. 

 

For me, the real issue with the old backup scheme in the organizer is that you can't easily 'reconnect' to the same folder structure like what is possible in Lightroom. I suppose that trying to apply the same workflow in the organizer would require a total overhaul of the catalog structure and management. However, I can imagine that with today's growing power of AI, Adobe could manage to improve its reconnection feature if the folder structure is exactly the same.

 

TOPICS
Organizer

Views

339

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Jun 10, 2020 0
Adobe Community Professional ,
Jun 10, 2020

Copy link to clipboard

Copied

trwalp,

The answer to your question is detailed in many of the posts I did those last days.

For backing up one's photo library, is it okay to use a separate backup program (e.g., Syncback), provided it retains the same folder structure as the original and retains all attributes and timestamps? Likewise, how about simply making a copy of the catalog in C:\ProgramData\Adobe\Elements Organizer\Catalogs?

Making a copy of the catalog folder does not work if you have to restore on a different drive (or drives) as the original ones. You can copy or move the catalog folder practically everywhere, but its contents don't change. All links stored in the catalog for the location of the pictures include not only the path in the folder trees, but the identification of the drive. If the drive is missing, all files in the copied catalog point to a missing drive. The catalog is unusable. Also, the present 'reconnection' tool in the Organizer can't reconnect a big library, contrary to what Lightroom can do.

 

So, any external backup tools will work only if the restore is done on the same original drive(s). Or at least with a new drive with the same identification, which would require a cloned drive, a Windows command to change the drive identification or a change of the contents of the Organizer with an Sqlite tool.

 

I prefer to avoid proprietary backup schemes because the results are inaccessible and unusable if corrupted in the slightest way. 

Just consider zipping your media folder tree: it's a valid backup scheme even if you don't understand what's in it. The real issue with the organizer catalog in this respect is that you can look into it and alter its contents. The backup folders contains

- a 'catalog.buc' file which is a renamed copy of the 'catalog.pseXXdb' database.

- renamed copies of each of the media files (including xmp sidecar files)

- a special file 'backup.tly' which stores lists of files to restore the folder structure and manage your backups, full or incremental. If this file is missing, that means your backup is corrupt. It's the last file written into the backup folder.

- other accessory data or database such as the thumbnails cache and face recognition and other data.

 

So, using an external backup tool can't work, and that's true for all catalog based softwares. 

 

For me, the real issue with the old backup scheme in the organizer is that you can't easily 'reconnect' to the same folder structure like what is possible in Lightroom. I suppose that trying to apply the same workflow in the organizer would require a total overhaul of the catalog structure and management. However, I can imagine that with today's growing power of AI, Adobe could manage to improve its reconnection feature if the folder structure is exactly the same.

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 10, 2020 1
Explorer ,
Jun 11, 2020

Copy link to clipboard

Copied

Another incredibly substantive reply, thank you. I'll have to find your previous posts to which you refer.

 

Re your revelation about Drive ID's: All links stored in the catalog for the location of the pictures include not only the path in the folder trees, but the identification of the drive."

 

Is the Drive ID referred to in this article, Changing the Disk ID of a Drive , the "drive ID" to which you refer? 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 11, 2020 0
Community Beginner ,
Jun 11, 2020

Copy link to clipboard

Copied

Michel, I either don't understand or don't agree with your comment that 'using an external backup tool can't work'. I have my catalog on my C:\Drive and have used Acronis True Image for years to do a daily backup. I have my Media on my NAS and have used the NAS' proprietary backup for daily backup for years. I have had to restore both at various times and am careful to make sure that the Media is restored to it's original location (and that the NAS share is mapped as the same drive letter on the PC with the same file structure. I have never had a problem when I restore. Am I missing your point?

 

trwalp, re your comment 'I prefer to avoid proprietary backup schemes because the results are inaccessible and unusable if corrupted in the slightest way' I can certainly understand although the same thing can happen with a zip file (or technically with a regular file though I can't say that has ever happened to me). The backup program I have (Acronis) has a validation app and my practice is to have Acronis automatically validate the backup file every time I create one. While I have had corruption problems in the past, I have not (yet) had one with the current version. I especially like the fact that you can schedule the backups in a number of different ways and, in my case, have the backup file stored on a NAS share. I don't know if other Backup apps have the validation feature but for the reason you cite, I wouldn't get one without it. Trying to recover a corrupted backup has got to be the most frustrating things that can happen.

 

Bob

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 11, 2020 1
Explorer ,
Jun 11, 2020

Copy link to clipboard

Copied

Thank you for the additional input, Bob.  

 

In this particular thread my goal is not to simply backup and restore to the same system and drives; it is to move the photo library and catalog to an external SSD that can then be used with PE2020 on two different PCs.

 

Since both you and Michel addressed my backup preferences I must add that I want a backup that looks identical to the original source, i.e, a Windows folder structure just like the original data. In some instances I use SyncBack's versioning system but it's pretty much invisible. I don't want my backups proprietary, incremental, zipped, nada -- because I can't simply connect the backup drive and access the files. Beyond Compare is my best friend, and it cannot run a compare against a proprietary backup! 

 

For drive images, Terabyte Image files (like those from Acronis) are fine, and I agree with you about the verification step.

 

Interesting that you use a NAS. As I wrote, that's how I started this project. But having used PE2020 for a few days I discovered the error in my plan. Stupidly, I thought my wife and I could share the photo library and catalog, which turns out to be impossible. Plus, there's latency with the NAS. I can hear it spin up when PE runs, which briefly freezes it, then when using PE there is the occasional delay that I attribute to the NAS.

 

Someday, someone smart is going to release a Picasa/Elements/Lightroom-like program designed for today's family that needs to frequently transfer photos from their phones to a central family library, and all have access to it for editing, printing, and creative uses.  This single-user design is from a by-gone era, where one person in the family was "the photographer".

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 11, 2020 0
Community Beginner ,
Jun 11, 2020

Copy link to clipboard

Copied

trwalp, I agree with you that it would be nice if Elements was made into a multi-user/multi catalog system. Not only is that archaic but there is a lot else in Elements that could use modernization. But I guess that Adobe sells products via new, Gee-whiz features so that's where they expend their resources.

 

Bob

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 11, 2020 0
Adobe Community Professional ,
Jun 11, 2020

Copy link to clipboard

Copied

Michel, I either don't understand or don't agree with your comment that 'using an external backup tool can't work'. I have my catalog on my C:\Drive and have used Acronis True Image for years to do a daily backup. I have my Media on my NAS and have used the NAS' proprietary backup for daily backup for years. I have had to restore both at various times and am careful to make sure that the Media is restored to it's original location (and that the NAS share is mapped as the same drive letter on the PC with the same file structure. I have never had a problem when I restore. Am I missing your point?

I also use Acronis, which has been useful several times to save me from issues on my system drive (on which I don't store catalogs or media files).

What I mean as explained in other threads, is that if your original drive holding your media files is completely unavailable and you have to restore on another drive, the links in the restored catalog still point to the original drive. Just try to do a restore on a replacement drive and all your files will be disconnected. Using an Acronis clone is a different case, the cloned drive gets the same internal serial number. In my experience in this forum I think it's very dangerous to feel safe because you have a NAS with its own backup system. Just try to simulate the disparition of your original drive for some reason and restore on another new drive. If that works and no files are disconnected, then I am wrong and you are welcome to correct me and to explain how exactly you can avoid to reconnect.

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 11, 2020 1
Adobe Community Professional ,
Jun 11, 2020

Copy link to clipboard

Copied

Someday, someone smart is going to release a Picasa/Elements/Lightroom-like program designed for today's family that needs to frequently transfer photos from their phones to a central family library, and all have access to it for editing, printing, and creative uses.  This single-user design is from a by-gone era, where one person in the family was "the photographer".

No new era, that is a dream that has been going on for twenty years. It's particularly present in Lightroom forums where so many pros would pay much, much more to get the ability for real workgroups. If such a dream had been easy to offer, we should have a lot of solutions available from competitors. It's not going to happen with Lightroom "Classic", and the new Cloud version is still much inferior in features and lacks compatibility to share keywords structures with the Classic version. Anyway, that's where the efforts of Adobe are targetted now, and certainly not with Elements. Sharing media has become the rule today with Cloud storage. Organizing and saving your photos for the future is quite another matter. What other existing and affordable design can you suggest to replace the single-user era?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 11, 2020 1
Explorer ,
Jun 11, 2020

Copy link to clipboard

Copied

Multi-user Quickbooks comes to mind. What's the difference between the catalog and a multi-user database?

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 11, 2020 0
Adobe Community Professional ,
Jun 12, 2020

Copy link to clipboard

Copied

Multi-user Quickbooks comes to mind. What's the difference between the catalog and a multi-user database?

Multi-user softwares are indeed everywhere now. The backup/restore issue is not if the database is multi-user oor not. It's about what is recorded in in the database. What is written into the organizer main database is the identification of the location of the file, which includes both the 'path' in the folder structure and the disk identification based on the internal serial number of the drive as well as the drive letter and volume name and type (internal or external). What is written is true... until you have to restore to another drive after the original is no longer available. That's the issue for backups.

Here is how the drives are shown in the sqlite database:

Volume_table-1.jpg

 

(By the way, I would be interested to see how the drive is identified in the case of media files on a NAS, specified by drive letter or UNC path). This is assuming you have a sqlite utility to display the volume_table of the catalog.psexxdb database. I have no NAS to test it and absolutely no need to use one.

 

You get the internal serial number of a Windows drive form the DOS command Vol X:, where X is the drive letter. You get the value in hexadecimal and it appears in decimal in the volume_table.

 

In the case of Onedrive or similar (Dropbox, Google...), what the organizer can manage is the dedicated folder which is synced to the cloud. That folder is referenced just like an ordinary folder. Imagine you have synced your Onedrive folder with the computer of your wife  as well as a copy of your catalog in Onedrive,

Your wife's computer has the same media files structure as yours, and the same catalog. (Copying the whole catalog folder is fast). The problem is that this catalog points to your computer drive, not the synced media folder tree in your wife's computer. All files will be disconnected, even with the same path and drive letter.

Note that instead of your wife's computer, it could be another of your own computers away from home (my second computer is 330 kilometers away.)

Is there a solution  to make your wife's computer or your second one recognize its own disk?

- reconnecting? Just try it and you'll see it's irrealistic for a normal sized library.

- fooling the organizer by using a DOS command to assign the same drive identification? That should work.

- using an sqlite utility to edit the volume_table utility with the same drive? That's what I do in a similar situation where I don't work on a synced copy on the Cloud, but on a synced copy on another external drive, via Microsoft Synctoy.

 

That shows that instead of doing a full backup and restore (mininum 4 hours for 75 000 items) I can use Synctoy to sync the catalog copy in a just few minutes and edit the volume_table. I could do the same with Onedrive. I think that it might be worth to test the Onedrive solution in the situation where each computer works with the library on a similar external drive with same letter and serial:

http://www.johnrellis.com/psedbtool/#_Change_a_Drive%E2%80%99s

That would be a valid sharing and backup solution if you find it ok to use a cloud-based solution.

I am happy with using:

- organizer backups from time to time (and always to change computer) to dedicated external drives.

- Microsoft synctoy after each session (a few minutes) to sync to a second external drive 

- my catalog and library on the first external drive which I can plug in instantly to my wife's own computer and license or to my second computer 330 kilometers away

- Acronis to backup automatically the system and programs drive

- an sqlite utility to update the serial number of the synced drive if needed.

 

Did I state that beside using alternatively my external drive with catalog and library, I also 'sync' them to the internal drive of my two computers as well as the one of my wife? Yes, I am a bit paranoid about safety.

 

 

 

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 12, 2020 1
Explorer ,
Jun 14, 2020

Copy link to clipboard

Copied

Michel, 

 

I am very interested in your most recent reply and will try to respond soon, after digesting and researching it.  But to clarify, the subject of the Catalog being multi-user is really the overarching issue that led me to starting this thread. I didn't initially use the term, multi-user, but if PSE provided the ability for families to share a photo library and catalog, my attempt to use a NAS would have succeeded instead of failed. All the ensuing questions would not have arisen.

 

Nonetheless, I must deal with cards as dealt. Here's the latest hiccup in trying to move from the NAS to a portable SSD. If my users can't access the catalog and photo library by sharing it, then the library and catalog must be made portable.

 

Thank you!

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 14, 2020 0
Adobe Community Professional ,
Jun 14, 2020

Copy link to clipboard

Copied

I have answered your question about the restored folder name in your last post.

I would really like to know more about what other users with NASs are doing and if there are differences between Macs and Windows. They all can share the media files between different users normally, but not the catalog unless I am totally wrong. 

If I had a NAS with my media library, I'd keep my personal catalog locally, and only do a copy of the catalog folder somewhere on the NAS, not to be used directly from there, but to be copied by each individual network user locally on his/her  own computer. The copy of my own catalog folder will take about 5 minutes (3.5 GB). Not really a better solution than storing the library and catalog on a USB3 external drive and plugging it alternatively to each computer. Same for storing the library on the Cloud with Onedrive or Dropbox.

 

 

Likes

Translate

Translate

Report

Report
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
Reply
Loading...
Jun 14, 2020 0