Organizer catalog to see My Pictures copied to external hard drive

Report · Jan 24, 2018

There are a great many discussions on this forum about using an external hard drive for photo storage. The more I look at them the more confused I become.

On my desktop I have PE 8, Windows 7.

Currently the Organizer catalog is set to use photos in MyPictures for the catalog. Of course, I have too many photos so an external hard drive is the answer.

Today I copied MyPictures to one external hard drive as a backup in case of problems using windows explorer. This takes about 5-6 hours. Then I copied MyPictures again to another external hard drive which is the one I would like Organizer to use. From what I read in the forum this is not the proper way to do this. I'm supposed to do all this within Organizer. Do I need to start all over, deleting the hard drive storage and using the backup and restore method within Organizer? I'm having a hard time understanding how that works and I'm afraid I'll mess it up. I haven't deleted MyPictures from the desktop hard drive yet. And I guess I will have to do that from within Organizer, too?

I guess there is no easy way to tell Organizer to look in a new place and also recover all the tags.

Sorry to ask about an issue that has been discussed many times but I am thoroughly confused.

Thanks.

Eva

Report · Jan 24, 2018

Eva,

Let's go step by step...

First, you have a big library (media files). Just to compare, your need twice the time I need to do the same copy to an external drive from the Explorer (about 500GB for 70 000 media files). So, it's important to choose an efficient (and safe) method. Your catalog folder is by default stored on the C: drive. You find its location and size from the menu Help >> System information. The catalog folder can be stored (copied or moved) anywhere else (it is shown in 'custom' location in the 'manage catalogs' dialog list); its contents are not changed.

You understand that the database in the catalog stores the information for drive and path of each item of you library and that the catalog is never aware of changes you are doing outside of the organizer.

If you use the explorer to move one photo file from you main drive to the external drive, the recording of the drive and path of that item is not updated. The organizer can't find any file at the old location; it shows the file as 'missing' or 'disconnected'. The other pieces of information in the record of the database are still unchanged, exif, tags, thumbnail. You have to tell the organizer to 'reconnect' that is to find the new location and to update the record with the new drive/path. If you know the new location, you can guide the organizer to it and the file will be reconnected. If you don't know, the organizer will start a search comparing the data in the catalog to the data of all other files present on your computer. Just do the test for a single moved photo file and you'll see how long scanning all your files will be...

If all your library has been moved to a new drive, or even if a folder has been moved, doing the scan for each disconnected file will be much too long if you don't help the search in the 'reconnect' dialog by guiding the scan to the new folder.

Photoshop Elements (PSE) knowledge base.

In your present situation, reconnecting is totally unrealistic. It would mean making all your files 'disconnected' on the master drive first, by deleting them or making them appear as such; for instance in renaming temporarily the master folder. Then the reconnect function will try to reconnect... and that will be too long even if you guide the scan for the new drive.

Now, what if you use the options of the organizer to move one file or a folder, or even the whole master folder of your library to the external drive? You set the left folders panel to tree view and you drag and drop the master folder to the desired drive. That's all. The organizer uses the explorer routines to move the files to the new location and at the same time, it updates the contents of the catalog. The move by itself takes the same time as your move from the explorer, and the re-indexing adds some time (negligible in fact). Nothing to reconnect. Your files are really 'moved', not 'copied', your master drive is clean. The process can be global, or folder branch by folder branch. The process is really safe, even if you have a problem in the move: I had once a power failure... Nothing lost, the files are not really deleted until the transaction is finished and successful. And you have your safety copy on another external drive. Just let the process run by night; there is no progress bar but the task runs ok even if the task manager says the organizer is not responding.

Now, let's have a look at the backup and restore process:

Photoshop Elements (PSE) knowledge base.

It's the safest process. It does take the same time as your copy process plus the move described above. After the restore, you have the restored folder tree on the external drive, plus a backup allowing restore on any other drive or computer with the links updated. You still have the original folder tree in your main drive. You delete it manually from the explorer when you are ready.

Just note that you have now a new catalog, also located on the external drive. If you prefer it on the default internal drive, the catalog manager has a 'move' button to move it back to default location.

Report · Jan 25, 2018

Thank you so much! A bit more clarification needed.

"After the restore, you have the restored folder tree on the external drive,"

I assume the restored My Pictures folder tree on the ext. HD "looks" like the one in Windows Explorer? So that I can access a folder or photo to copy or look at with Irfanview or the like? I know better than to delete a photo there because Organizer will scold me with a ? the next time I open Organizer. I like the fact that this method keeps My Pictures on the desktop HD until I'm sure it's safe to remove it.

"plus a backup allowing restore on any other drive or computer with the links updated."

Does that mean an additional My Pictures on the ext. HD? What does that backup look like?

I am an original version worrywart. I need my photos in two places just in case. What I have been doing for years is copy My Pictures from the desktop HD to an external HD once a month. What will I do once I have migrated to the external HD using Backup and Restore and deleted My Pictures from my desktop? I still need to have the photos in 2 places.

Would having the catalog on the desktop make Organizer run a bit faster?

Obviously, I'm having a hard time thinking this process through clearly. Your clear help is very much appreciated.

Eva

Report · Jan 25, 2018

I assume the restored My Pictures folder tree on the ext. HD "looks" like the one in Windows Explorer? So that I can access a folder or photo to copy or look at with Irfanview or the like?

Yes. It looks like exactly like the My Pictures folder, same hierarchy. Only difference: only the media files present in your catalog will be restored. That's the way to migrate the folder tree(s) to another drive or computer together with the catalog folder and to keep the links updated. A very common situation when you upgrade your computer or your drives.

I like the fact that this method keeps My Pictures on the desktop HD until I'm sure it's safe to remove it.

Yes.

"plus a backup allowing restore on any other drive or computer with the links updated."
Does that mean an additional My Pictures on the ext. HD? What does that backup look like?

No, the catalog backup folder is very special. It contains everything to re-create the image files folder tree and the catalog folder; mainly all your media files are renamed starting with "B00...", with all the other components of the catalog and a special backup.tly file to record how to restore everything in the new chosen destination. It's not a simple copy, you can't use it as such, only the 'restore' command can use it. That means the backup folder is generally created on another external drive and restored on your new destination, new computer, drive or partition.

Suggestion:

Create a new catalog, import a test folder. Do a backup on an external drive and look at the contents of the backup folder.

What will I do once I have migrated to the external HD using Backup and Restore and deleted My Pictures from my desktop? I still need to have the photos in 2 places.

You don't need to delete the photos on your computer. You'll have two independent "libraries", each one with its own catalog.

Just be aware that they are independent. What you do in one is not 'synced' with the other.

Would having the catalog on the desktop make Organizer run a bit faster?

Theoreticall yes, but it's not decisive if you have enough RAM. You should be able to compare if you keep your original library and catalog.

Report · Jan 26, 2018

Okay, I did as you suggested. I ended up doing it backwards but it worked.

I created a new catalog which I named "Family 2011 test".

I imported the tree corresponding to Family 2011.

Chose backup catalog: full backup

Copy all items from current catalog.

I chose as destination Ext HD J: (I should have created a folder but didn't).

Backup finished.

As you predicted, looking into J: I found B000 files.

Then I went back to PSE File and chose restore from J:. There was only a file name Backup.tly. I chose that to restore to ext. HD K: It worked. I opened the file on K: and the folder with all subfolders is there as it looks in Explorer.

Now I notice on the desktop I'm seeing only that tree. So as you said, the original stays and is still in explorer.

I should have put the backup file on HD K and the restored tree on HD J: So my next question is:

How to I remove the whole operation. Do I do that through PSE so that the new catalog is removed and the backup files from both HDs?

Now the point of this exercise is to move My Pictures to the external drive. Since the original photos on the desktop hard drive are not touched I won't worry too much about messing up.

But being a worrywart, I need a second copy on another external HD. Drives do fail. How do I go about that?

Do I do second backup and recover operation? And delete My Pictures from my desktop hard drive?

Since I will be using 2 external hard drives is it okay to have the backup file and the photos on the same HD?

Thanks again for your help!

Eva

Report · Jan 27, 2018

I created a new catalog which I named "Family 2011 test".
I imported the tree corresponding to Family 2011.

My suggestion was only intended to let you try the backup/rstore process on a small catalog, not to 'import' all your files in a new catalog. If you did try on a small test folder, you can simply delete the test catalog, the test backup folder and the restored test folder tree. Then you do the real complete backup (on your chosen drive) and the restore (on your chosen drive).

If you imported your whole image folder tree, you can do the same, deleting the test and starting from the backup. However, look at the present situation:

- when you imported your whole image folder tree, you recovered only the tags and captions you may have 'written to files' before. So chances are that tags are missing; anyway, importing does not recover your albums, stacks or version sets. The backup folder as well as the restored tree don't contain all your organization. Only the backup and restore or the 'convert catalog' process can keep all the information. Importing into an empty catalog may be necessary if the catalog has been lost or just for a test.

- Even if the correct folder tree is restored and you don't mind losing your albums and more, the backup folder and the restored file tree are on the wrong drives.

Even if that takes time, I suggest to delete as explained above, then do the backup and restore process paying attention to the destination drives.

- If you don't delete before, you can do that later if your external drives are big enough, but beware of not mixing up the good and bad folders and naming the backup folder and destination folder with other names.

- A catalog backup folder may be copied or move elsewhere from the explorer if you want (just like any catalog folder).

- As explained before, the image folder tree itself must be moved by the organizer itself or restored from a backup and restore process.

Report · Jan 27, 2018

Thanks again!

I removed the test catalog. I deleted the backup B000 file from the one ext. HD and on the other ext. HD I deleted the file with the photos. So now I'm back to square one and ready to try my luck with the full backup-restore of MyPictures.

My plan is to put the backup and the photo folder both on the same 4TB ext. HD. When and if that succeeds, I plan to use explorer to copy MyPictures which still reside on my desktop HD to a different ext. HD which still has room. That way I hope to reduce my worrywart anxiety about possible loss of my photos.

At that point my photos will still be on the desktop HD and I may need further instruction on how to remove from the internal HD and use PSE from the ext. HD.

Does this sound reasonable?

Eva

Report · Jan 27, 2018

Does this sound reasonable?

Yes, it is.

I forgot to comment on your question:

Since I will be using 2 external hard drives is it okay to have the backup file and the photos on the same HD?

That's not the best solution theoretically.

- copying or restoring on the same HD is slower than from two different drives (not critical)

- beginners often mix backup and restore folders...

- If the external drive fails, you lose both your backup and the restored files. But you would still have your original catalog and mediat files on the main computer. With another copy on a different drive you are still safer, so don't worry too much.

I would also consider the future.

You now have the opportunity to work either with the old catalog and library or the new catalog and library on the external drive. I imagine you will choose the internal solution to add photos and manage your tags and organization. The external solution will stay as two different backup solutions but they won't be updated. Periodically, you'll want to update your backups. If you have a lot of free space, no problem, you store the new backup folder where you want without replacing the old backup folder. You can even do a restore on a new master folder where you want. If space is become scarce, try to keep at least two different backup files from different times. I feel very safe if I have two different backup folders in two external drives, one at home and the other elsewhere (holiday home, or at my children...) Periodically I replace the older backup by a new one. The idea is that if there is theft, fire or the like, I have a good backup.

Another advice:

You can see that the backup and restore procedure is really safe, but its drawback is that it takes time. For that reason, backups tend to be unfrequent and you risk losing recent updates to your library in case of a crash. I don't like the 'incremental backups' from the organizer. Still too slow and too many users don't even test a restore and don't succeed in the process. I do use another solution: the Windows 'SyncToy' program.

I sync my photo file library as well as my catalog folder to an external drive. Once it has been synced once, it's very fast to update after each editing session. Only the changed and new files are updated which explains the speed of the process. Note that the 'synced' catalog folder refers to the original location, not the 'synced' folder tree. Not as good as the organizer backup, but valid to restore everything on the internal drive.

My choice:

- Synctoy to external drive A after each significant session

- full backup every month or two months, alternating on two other drives

- swapping those drives when

Report · Jan 28, 2018

I spent much of yesterday doing the backup and restore and putting a copy of MyPictures on another external HD. That took a lot of time. So here's what I've done:

1. Copied MyPictures from desktop HD to external K using explorer.

2. In PSE chose Backup to external J (full backup) 537GB. When that completed I looked and there was MyCatalog Backup with the B0000... files.

3. In PSE chose Restore from hard drive J:\My Catalog Backup\backup.tly. Restore to New Location J:\My Pictures EXT. I checked restore original folder structure.

There were two files that could not be restored B0037513.jpg and B0038339.jpg. I was asked if I wanted to continue and said yes. "The Restore completed successfully". I found the file in J and it looks to be the same as on the desktop HD with its tree.

4. At this point PSE is still open. The folder view is unchanged. File Catalog shows

Custom Location checked: J:\My Pictures EXT. And below MyCatalog [Current]. At the very bottom of the screen left it says "My Catalog". I don't know if that references the Catalog on the desktop or the Catalog on the External HD. Since both are identical I don't really know which one I'm seeing.

5. File Catalog accessible by all users shows My Catalog.

-------

6. The object of this whole operation was to get the photos off the desktop internal HD and onto an external HD. So I don't know what to do next. I need to remove photos from the desktop HD. And I need to be able to open PSE Organizer with the catalog and photos now residing on J. It should end up being the only catalog. Perhaps accessible by all users?

I think (I hope) I'm getting near the end . And thanks again for your patience with me.

I'll look into SyncToy next time.

Eva

Report · Jan 28, 2018

4. At this point PSE is still open. The folder view is unchanged. File Catalog shows
Custom Location checked: J:\My Pictures EXT. And below MyCatalog [Current]. At the very bottom of the screen left it says "My Catalog". I don't know if that references the Catalog on the desktop or the Catalog on the External HD. Since both are identical I don't really know which one I'm seeing.

The path shown is the correct location on J:

The name of 'My catalog' is the result of the restore.

I suggest using the catalog manager, 'Rename' button, and changing the name "My catalog' to "My Pictures EXT" or something else.

- nothing is changed on your C: drive

- you now have the restored library and the restored catalog on J: It is shown under 'custom location'.

When you use the catalog manager to open that catalog in the custom location, you can see in the information panel that all your files are seen as being in J:

(There is also another way to access the catalog on J: for another computer with the same Elements version. You simply double click on the catalog.psexxdb file, that opens the organizer with the catalog.)

5. File Catalog accessible by all users shows My Catalog.

That's the original catalog on C:

6. The object of this whole operation was to get the photos off the desktop internal HD and onto an external HD. So I don't know what to do next. I need to remove photos from the desktop HD. And I need to be able to open PSE Organizer with the catalog and photos now residing on J. It should end up being the only catalog. Perhaps accessible by all users?

You are already there

Just play with the new catalog and library until you are confident that all is well.

Removing photos from the desktop HD and the original catalog:

- to remove the catalog on C:, select the (current) catalog on J: There is a 'remove' button to remove an highlighted catalog. (You can't delete the catalog you are using currently, you must be in another catalog).

- Then use the Windows explorer to delete the media files folder tree on C: (don't worry, you have double safety with J and the backup)

About the choice of the location of the catalog, on C: or J:

- It is now on J: More space on C: It is available if you plug in the J: drive on another computer with the same PSE version (a secondary laptop for instance)

- or if you use the catalog manager and the 'move' button, you can move the catalog from J: to C: and it will be available for all users registered on your Windows computer. Slightly faster operation, less gain of space on C:

There were two files that could not be restored B0037513.jpg and B0038339.jpg

Don't worry too much, but check if they are available and not corrupt on the C: drive before deleting.

Report · Jan 28, 2018

Thanks. I'll wait a few days and then do the final step.

Eva

Report · Jan 30, 2018

Done, I think. Within the new catalog on J: I removed MyCatalog from C:. I then moved the current catalog on J: to Catalogs accessible by all users. My EXT catalog [current] now shows in the box. I suppose I should rename it since it isn't EXT anymore but the name reminds me of where my photos are. Then in explorer I deleted MyPictures. Actually, MyPictures remains but is an empty folder. So I think I'm almost finished.

I noticed that My EXT catalog is no longer on J:. But I can't figure out where it is now?

Also, do I have to do anything in the editor? Or is that unaffected by the backup|restore? Where are all the files related to the editor?

I sure got a lot of space back on my C: drive! I hope it functions a bit faster now.

Thanks again,

Eva

Report · Jan 30, 2018

This looks very good!

I noticed that My EXT catalog is no longer on J:. But I can't figure out where it is now?

When you move a catalog folder to 'accessible to all users', you move it to the default location on the main drive (in the Appdata folder). This is a hidden location. In Windows, you have to set the explorer to 'show hidden files and folders' to see it. You can see where any open catalog is situated from the menu Help >> System Info...

Also, do I have to do anything in the editor? Or is that unaffected by the backup|restore? Where are all the files related to the editor?

The editor is just that: an editor. It does not contain nor keep tracks of your files (except for the list of recent files).

Edit:

About the 'My Pictures' folder: it's a default folder for Windows. My guess is that if you delete or move it, it it will be created again.

I never choos My Pictures as my master folder for my photo library. I have my own master folder just under the root.

Report · Jan 31, 2018

More thanks!

The folder \appdata\Roaming\adobe\Elements Organizer\Catalogs\ is empty. But Help|Systems Info led me to Catalogs Location: C:\Program Data\Adobe\Elements Organizer\Catalogs\My EXT Catalog. Catalog size 198.9 MB. Catalog Cache size 1.1GB.

Now, yet another question. I try to back up PSE files once a month. More often if I manage to take a lot of photos. I know where My Pictures EXT is located on J: so I can back that up to K:. What else do you back up and where do I find it?

Thanks,

Eva

Report · Jan 31, 2018

Eva,

Now, you are safe with both your media files and catalogs.

Even in the event of the disparition of your computer, you can start again by reinstalling your operating system and programs.

You may have external resources like plugins,scrapbooking assets etc. Be sure to get copies somwhere, even if you can download and reinstall them later.

You global backup strategy with your computer will also depend on your other uses.

Your OS or your computer supplier generally have a solution to create boot up DVDs or thumbnails if some problem prevents you from booting. Take the time to create those safety DVDs.

What may take time in case of a big problem is to reinstal and to update the OS (hours for Windows...)

My global choice has been:

1 - to keep my personal data (including photos and catalogs) on a different drive or partition than the OS and the programs, the default C: drive.

2 - I have a 'cloning' and backup software (Acronis) wich permits saving a full image of the C: drive. Note that if you keep your catalog location by default on C:, they will also be covered, as well as other program preferences. This is important after installing new programs, significant updates for the OS or softwares or new devices. The backup time for C: will depend on the backup media (half an hour to one hour for me). Similar backup/cloning softwares can also deal with your media files and catalogs... with the problem that they don't guarantee to restore without 'disconnected' files.

Report · Feb 01, 2018

Here is my plan regarding backup of PSE:

Back up My Pictures EXT from J: to K:

I looked around my computer for "Adobe" folders. And copied those to K: as well. They are:

C:\Program Data\Adobe\Elements Organizer, the folder which contains My EXT Catalog.

C:\Eva\appdata\Roaming\Adobe\Photoshop Elements, the folder contains folder 8.0 which in turn contains editor plugins and presets.

C:\Program files (x86)\Adobe\Photoshop Elements 8.0

C:\Program files (x86)\Adobe\Elements Organizer 8.0

I don't know if they would be useful in case of Photoshop Elements corruption on my desktop.But as long as I have room on K: I think I'll copy them there.

I do have an older version of Acronis on my desktop. Several years ago with a previous computer I heard some scratching noises coming from the hard drive. Very quickly I successfully made an image backup. The HD indeed died a few hours after. With a new hard drive installed I restored the image! Whew! A couple of years later my laptop died. I had an image but it was a bit old. I used it to restore but was only partially successfull. Too much had changed in the meantime. But I do continue to make an Acronis image of the desktop once a month along with saving important files.

Do you have any comments?

None of this would have been successful without your patient help!

Eva

Report · Feb 01, 2018

Eva,

Your question is a very important one and this discussion should help many other users and hopefully prevent serious issues or even disasters.

Let me try to summarize for other users:

*-----------------------------------------------------------------------------------------------------------------------------------------------------

1 - before starting migration of pictures or catalogs... Backup from the organizer - read about migration and even ask here before.

2 - Storing your image files and catalog on an external drive will be more and more frequent. It's a good solution.

3 - Users never think about how to migrate from computer or drive until it's necessary, they rarely test the restore functions.

4 - Frequent now: installing a new small SSD drive for OS and hardware and keeping the old drive internally or externally. Do a backup first anyway. Either move your image folder tree to another external drive before, or use the restore process on 'custom destination' after the installation on another drive. Beware of cloning utilities which assign the same internal serial number to two drives: that will fool the organizer.

5 - catalog folders can be moved or copied.

*---------------------------------------------------------------------------------------------------------------------------------------------------------

As you can see, that generally require two or three external drives... one for the backup folder, one for the destination folder and preferably another one to keep in a different place outside your home in case of theft, fire...

Organizer catalog to see My Pictures copied to external hard drive

1 Correct answer