Moving "Pictures" and PSE catalogs to a new hard drive - after the fact!

[Welcome, Martyn]

My hard drive failed recently. I thought I had adequately prepared for this by maintaining backups to my NAS, but apparently not. I used the NAS' own backup software, but now I've discovered from reading this forum that restoring from the NAS to a new HDD won't work because the HDD will have a different serial number (*sigh*). I think I have a solution, but it involves quite a lot of work so I'd like to solicit some expert advice first.

My "old" computer had a small SSD cache and a large HDD. The HDD was the (C:) drive and it was the SSD that failed. My rebuilt computer has a larger SSD for the Windows operating system and a very large SSD for data. The OS has to be on (C:) and the data SSD is currently (E:).

Here's my plan: the original HDD (C:) still works, so I'm going to reformat it, put it into a spare PC, and install Windows on it. I will then copy "Pictures" and the PSE files to it from my NAS. I will then open the PSE catalog on this spare PC from PSE which is installed on my rebuilt PC via my home network. I hope that I will then be able to do a PSE backup either to my big, new SSD or to an external drive (I don't think it matters). Finally, I will do a PSE restore to my big, new SSD.

Questions:

1. Will this work and will all my tags and captions appear correctly?

2. Is there an easier way? My "Pictures" library is much to big to re-import and re-tag.

My thanks in advance for your wisdom!

[MichelBParis · Accepted solution]

Hi @Welcome, Martyn 

It's always much more difficult to solve a similar problem "after the fact", that's quite right.

Just note that if I had to give some advice "before the fact", that would include:

- don't use a NAS for storing your pictures and catalog databases. The reason is I am not competent... Adobe will not support NAS issues and successful NAS users never come here to help their friends. Use your NAS instead for backups, for instance.

- Just like Lightroom users, ignore the default storing of your pictures in the default Windows 'Pictures' system folder. Ideally store them on your own master folder directly under the root drive of an external or internal drive (or a new partition).

- It's good to have another backup solution than the integrated one in the organizer for both your pictures trees and your catalog folders. I believe that's what you have now. You are not sure to restore your library with no 'disconnected' files, but that may help.

I don't understand precisely your project but I have a feeling it will fail to reconnect. As you have already read, your catalog stores the location of your media files by identifying the internal Windows serial number for the drive as well as the full folder path on that drive. I believe that choice was absolutely necessary to distinguish between removable media like CD/DVDs as well as to provide a solution for the way Windows attributes drive letters. This allows recognizing a drive even if Windows has attributed the same letter to another device inbetween (not guaranteed...)

The major drawback is that nobody knows how to get that serial number. To get it, you have to start a DOS command (execute / cmd) wich opens a text window in which you type:

Vol X:

for the X drive for instance.

The serial is returned in hexadecimal form.

You can translate that hexadecimal number into decimal with the 'Calc' tool in programmer mode.

Advanced users familiar to the Sqlite database management can open the main database 'catalog.pse19db' and look at the volume_table which shows the recognized drives by the catalog with their properties. They can find the record for the drive with its properties from the initial drive letter. They can assign the properties of the new drive (letter, serial, type...) to replace the old ones, without changing the internal identification of the drive by the database. That only requires knowing the serial number of the new drive.

I use this myself instead of backup and restore betwen several drives. I use a fast Microsoft Synctoy utility to sync my folders tree between two drives. A few minutes instead of three hours backup and as many for the restore. Then a couple of minutes and I am ready with the updated catalog on the new drive.

And for most PSE users not acquainted with sql language?

A possible solution can be tried from this help file for the psdbtool created by John R Ellis:

psedbtool (Photoshop Elements Database Tool)

scroll down to:

Change a Drive’s Volume Serial Number

So, if you have an exact copy of your library on a different drive and you know the new serial number, you can make the organizer recognize the new drive instead of the old one. However, you should not have two drives with the same serial at the same time. That's one more reason to store your library on a different drive than your (C:) system drive.

Special notes on cloning drives.

If your library is on a different drive, the clone will work just like the original. Cloning will keep the serial number.

However, it's increasingly common to replace a classical drive for your system (C:) drive with a cloned SSD. If your pictures are on (C:) and the SSD is big enough to hold your whole library, OK. If it's too small and you want to use the original (C:) drive under a new letter your catalog will see two distinct drives with the same serial and the catalog will be corrupted. Better to move your photofiles to a different drive before replacing the system drive with the SSD. If you have a correct backup, you'll be able to move the catalog and library normally. Another good reason to keep them on a different drive than the system one.

Tip for advanced users knowing sql: if you have no idea of the serial number of the unavailable old drive, it's in the volume_table if you have a copy of the catalog folder, even an old one. Let's say you want to edit a copy of your catalog dabatase for a drive to work with another drive. Locate the original drive from its letter. If the new drive is already in the table under another letter, you can check by its serial and delete the whole record. Then you change the properties (mainly the serial and letter) in the old record. That's all.

[Welcome, Martyn]

Thank you for taking the time to write a detailed reply, Michel. Since posting my question, I have tried several simple things - typically copying my pictures and PSE files to various places. To my surprise, with the pictures and files on my new SSD C: drive, PSE is working normally, complete with the correct links to the image files, and correct tags and captions. This is without attempting to change and serial numbers. I don't know why this happened, but it's working!

I agree with your statement about using a separate drive for the image files. I have since removed my original HHD and have re-labeled my second SSD as drive D:. The question now is how do I move my image files and PSE files to this D: drive and still have everything work?

I can understand John Ellis' instructions for changing serial numbers, but the sqlite stuff is beyond me. Would I (i) make a note of the S/N of my C: drive, (ii) change it to something else, (iii) change the S/N of my D: drive to the origianl S/N of my C: drive, and (iv) move the files to the D: drive?

Alternatively, would a PSE backup from the C: drive followed by a PSE restore to the D: drive work and be more foolproof?

I appreciate your help with this; I think I'm nearly there!

[MichelBParis]

Thank you for taking the time to write a detailed reply, Michel. Since posting my question, I have tried several simple things - typically copying my pictures and PSE files to various places. To my surprise, with the pictures and files on my new SSD C: drive, PSE is working normally, complete with the correct links to the image files, and correct tags and captions. This is without attempting to change and serial numbers. I don't know why this happened, but it's working!

By @Welcome, Martyn

What I can say is that if you copy a given folder tree of your pictures to another location (drive, partition, other master folder...) the links in your current catalog are not updated. The same catalog will only recognize the new copies if they have the same properties (drive letter, serial number and hierarchical path). That's true if you have made a 'clone', which copies those identifiers to the new drive. Otherwise, the double drive identification, letter and serial is supposed to help the organizer to recognize a given drive from its serial if Windows has changed the letter drive. I don't know any other way to have the same catalog recognize copies of the file folders made externally of the organizer. Of course, folders moved internally by the organizer do update the catalog. As already mentioned previously, a catalog can't store two different drives identifications with the same serial or drive letter without being corrupt.

I agree with your statement about using a separate drive for the image files. I have since removed my original HHD and have re-labeled my second SSD as drive D:. The question now is how do I move my image files and PSE files to this D: drive and still have everything work?

You can move whole folder trees from one drive to another from the organizer left folder panel by drag and drop. This updates the links for the new drive identification.

I can understand John Ellis' instructions for changing serial numbers, but the sqlite stuff is beyond me. Would I (i) make a note of the S/N of my C: drive, (ii) change it to something else, (iii) change the S/N of my D: drive to the origianl S/N of my C: drive, and (iv) move the files to the D: drive?

Alternatively, would a PSE backup from the C: drive followed by a PSE restore to the D: drive work and be more foolproof?

Yes, using the Windows commands to retrieve and possibly change the serial number are a possible solution without any sql tools.

A full backup and restore is always recommended and more foolproof, that's right, but it may take more time and there are unfortunately a few cases where the backup/restore does not work. In that situation, if you have a good external backup at your disposal and you know the serial number expected by the catalog, then attributing that serial number to the new drive can solve the problem and the full location for each media file: drive+path will be the same in the catalog and in the new drive.

[Welcome, Martyn]

Thank you again. I've read several descriptions of dragging and dropping from a folder tree within PSE, but I think this must be a feature that was added to more recent versions than mine. I'm using PSE 10 and haven't been able to find a way of doing this. There is a command: File > Copy/Move to Removable Drive, but I don't know what this is intended to do and I have been reluctant to try it.

[MichelBParis]

Yes, the way folder trees were shown in PSE10 was drastically changed in the following PSE11. However, I just had a look to verify that you could move folders (with their subfolders) also in PSE10. You have to use the display menu to 'folder location' and you can drag and drop a folder from a drive into another.

The Copy/Move to Removable Drive (Ctrl Shift O) is  the menu to get the same move ability (or the copy). At least in recent versions, that old legacy command also works to internal drives as well.

Before moving a big folder tree, check if that works as expected in your old version with a small folder tree... that's a long time I used PSE10.

[Welcome, Martyn]

Using the Display > folder location method didn't work well. PSE seems limited to moving only small folders (MBytes instead of GBytes), and I couldn't get Copy/Move to work acceptably either. However, I was successful in doing a PSE backup from C: to my mostly empty D:, and then restoring to a folder immediately below the root D:. All my metadata were intact. PSE brought my catalogue folder along as well.

I had created a folder D:\Photographs as the destination for the restore. Unfortunately, PSE decided to add a lengthy path to this, which makes it all rather messy. I ended up with D:\Photographs\Users\[Name]\Pictures\. . . I might be able to simplify this by linking back to Libraries, but that's for another day. At the moment, I'm just pleased to have my photos on my data drive and fully connected.

Incidentally, the PSE backup from one SSD to another took around four minutes for 8,000 photographs! The restore was much the same, although PSE then took around 20 minutes to regenerate the thumbnails.

Thanks again for your help, Michel.

[MichelBParis]

Using the Display > folder location method didn't work well. PSE seems limited to moving only small folders (MBytes instead of GBytes), and I couldn't get Copy/Move to work acceptably either. However, I was successful in doing a PSE backup from C: to my mostly empty D:, and then restoring to a folder immediately below the root D:. All my metadata were intact. PSE brought my catalogue folder along as well.

By @Welcome, Martyn

Not a surprise: the folder management for older versions up and inclusive to PSE10 was awkward and buggy. PSE11 introduced a new concept in which the folder management was primarily constructed from the locations existing in the catalog.

I had created a folder D:\Photographs as the destination for the restore. Unfortunately, PSE decided to add a lengthy path to this, which makes it all rather messy. I ended up with D:\Photographs\Users\[Name]\Pictures\. . . I might be able to simplify this by linking back to Libraries, but that's for another day. At the moment, I'm just pleased to have my photos on my data drive and fully connected.

By @Welcome, Martyn

The problem is not in PSE, it's in Windows. The lengthy path is the real one in Windows, and the paths starting with 'Pictures' are Windows shorcuts. That's why it's never a good idea to store one's photos in the Windows 'Pictures' file. Note that with recent Organizer versions, you can move the useful part of the folders tree up in the drive, just under a new master folder you create under the root. The drag and drop works there. A full backup before the drag and drop would be a good idea for safety.

Very glad you were able to solve your problem.