Planning a large photo archive

[jsmcwilliam@gmail.com]

Hi,

     I run PSE 2024 under Windows 11 and have several thousand images to be archived. The work has begun but I need confirmation that my approach is optimal. I have read the reports of how users loose their data often following corruption of enormous catalogue files. My approach has been to work with smaller folders and catalogues in the following way:

1. My laptop has limited memory so I decided to save my images to an external flash card.

2. I created folders on this card for each year from 1960 and transfered the images accordingly.

3. I imported the images to PSE in batches of 10 years with a catalogue for each 10 yr. period. I sorted and tagged each batch.

4. I created backups of the sd card as my work progressed.

I have now catalogued about 25% of my images and am asking whether this strategy would appear to be optimal before continuing.

John McWilliam

[Greg_S.]

I'm not sure what you consider a large photo archive to be.  IMO, the optimal approach is to have all files in a single catalogue.  I currently have over 250k files in mine.  Elements 2024 has a backup feature that will automatically backup your active catalogue to your system drive.  (I don't think you can change the location of the backups.)  The problem with using several different catalogues is that only the one that is active at the time you close the Organizer will be backed up.  And since I believe only 3 or 4 backups are retained at a time, the possibility of automatic backup of your many catalogues will likely be impossible, and you could easily overwrite necessary backups.  This may not be important to you since you seem to have alternative backup plans in place.  But it is certainly a consideration.

You mention that you have seen reports of data loss from corruption of a catalogue.   Any such reports are extremely rare and involve some form of user error or hardware failure.  A good backup plan will avoid those issues.

When you say your laptop has limited memory, are you talking about storage space or RAM (or both)?  If you are running short of hard drive space, you can move the catalog folder (which contains most of the data information) to an external drive or flash card.  BTW, what is the size of your flash card?  My preference would be to use an external hard drive where I could store both the images and the catalogue or a backup of the catalogue. 

[jsmcwilliam@gmail.com]

I understand your argument for one catalogue with all the images. As you are now aware I have seven, one for each decade from 1960.
1. Is there any way I can merge these without loosing information?
2. Is the information (name tags, place tags etc.) also contained in the metadata?
3. Am I bound to the worst case senario and have to create a new caralogue from scratch?
John

Sent from my iPhone

[Greg_S.]

You have told me that you have 7 catalogues, but you haven't answered my other questions.  I also have some additional questions which may affect my advice to you.

1. Approximately how many photos do you have in your 7 catalogues?
2. How much free space do you have on your laptop's hard drive?
3. How much RAM do you have?
4. Do you have more than one external drive or flash drive?  How many and what size?
5. Have you used facial recognition on all of your images?  If so, will it be sufficient for you to know just the names of the people in the images?  Or do you need to know that the third person from the right is Great Uncle Harry?
6. Have you created any Albums in the Organizer and, if so, have you customized the order of the images in the Album?
7. Have you used the system Places and Events tags, or have you only used keyword tags?
8. Have you saved the metadata to the files in the Organizer?  (Ctrl+W)

The basic answer to your question is that you will be able to combine the catalogues without losing any information you have added in the Organizer.  But depending on your answer to my questions, there may be some minor exceptions.  Even in those cases, there are some fairly easy workarounds.  So, you should not be forced to start from scratch.

[jsmcwilliam@gmail.com]

Re: Planning a large photo archive

I'm not sure what you consider a large photo archive to be.  IMO, the optimal approach is to have all files in a single catalogue.  I currently have over 250k files in mine.  Elements 2024 has a backup feature that will automatically backup your active catalogue to your system drive.  (I don't think you can change the location of the backups.)  The problem with using several different catalogues is that only the one that is active at the time you close the Organizer will be backed up.  And since I believe only 3 or 4 backups are retained at a time, the possibility of automatic backup of your many catalogues will likely be impossible, and you could easily overwrite necessary backups.  This may not be important to you since you seem to have alternative backup plans in place.  But it is certainly a consideration. You mention that you have seen reports of data loss from corruption of a catalogue.   Any such reports are extremely rare and involve some form of user error or hardware failure.  A good backup plan will avoid those issues. When you say your laptop has limited memory, are you talking about storage space or RAM (or both)?  Laptop: 370GB used + 103GB free. Ram: 8GB If you are running short of hard drive space, you can move the catalog folder (which contains most of the data information) to an external drive or flash card.  BTW, what is the size of your flash card?  USD flash memory: 19GB used + 38GB free. My archive to date contains roughly 15% (15GB) of the total number of slides I want to scan and catalogue. My catalogues to date (7) are approximately 0.5GB (4GB). I expect that there is a lot of duplicated info there and a single catalogue would be more effective regarding space. My preference would be to use an external hard drive where I could store both the images and the catalogue or a backup of the catalogue.

[jsmcwilliam@gmail.com]

Re: Planning a large photo archive

I hope this way of answering your questions is acceptable. You have told me that you have 7 catalogues, but you haven't answered my other questions.  I also have some additional questions which may affect my advice to you. Approximately how many photos do you have in your 7 catalogues? (4000). How much free space do you have on your laptop's hard drive? (103GB). How much RAM do you have? (8GB). Do you have more than one external drive or flash drive? How many and what size? I have 4 USB ports and a SD micro port. [I also have a desktop with 3 USB and 1 standard SD port. It has plenty of space so I have a backup of my images saved there.] I have a number of USB memory possibilities. At present I am using a 57 GB unit where I store all the images and a backup of my catalogues. The only reason for using the Laptop is due to the fact that my scanner runs on a Hamrick driver which is installer there as is my copy of PSE. In retrospect it would have been better to run everything from the desktop. Have you used facial recognition on all of your images? (most of them). If so, will it be sufficient for you to know just the names of the people in the images? (Yes).  Or do you need to know that the third person from the right is Great Uncle Harry? Have you created any Albums in the Organizer (Yes, one for each year) and, if so, have you customized the order of the images in the Album? (Yes). Have you used the system Places and Events tags, (Yes). or have you only used keyword tags? Limited: I only use keyword tags for things like “Miscellaneous, Workshop, Music, Documents”. Have you saved the metadata to the files in the Organizer?  (Ctrl+W). (Yes). The basic answer to your question is that you will be able to combine the catalogues without losing any information you have added in the Organizer.  But depending on your answer to my questions, there may be some minor exceptions.  Even in those cases, there are some fairly easy workarounds.  So, you should not be forced to start from scratch.

[Greg_S.]

Thanks, John, for being so thorough with your responses to my questions.

So, I think if I were you, I would create a new catalog and import all of the files into that new catalog.  Here is what should happen, as well as some workaround suggestions to preserve as much of your cataloguing work as possible.

• If something doesn't turn out as you expect, you will still have your existing catalogues available to work with and to try other methods of consolidation.
• Your Albums will not be preserved.  However, it should not be too difficult to recreate them or to use a different method to meet your needs. 
• If you have dated all of your scans so that they appear in chronological order when you use the Sort by: Newest or Oldest filters, there should be no need to separate them in Albums.  You can either scroll through the grid chronologically to find photos for a particular date or year, or perhaps use the Timeline (Ctrl+L), and its calipers at each end, to limit your view to a particular time frame:

• 

• You could select each of your current Albums and apply a new keyword tag to them.  For example, since your Albums all seem to be for a particular year, create keyword tags for that year, select all images in an Album (Ctrl+A) and apply the corresponding tag.  After saving the metadata to the files, the new "album" tags will be imported into the new catalog when you import the files.  You can then reverse engineer the Albums by creating one for each tag year, and dragging the applicable Album icon to each year's photos.
• People tags that you have created through face recognition, will not be transferred when you import the images into the new catalogue.  Instead, keyword tags will appear for each of the People names that have been recognized.   These will appear under the new catalogue's Imported Keyword Tags category in the Tags panel.  The recognized faces will not be displayed in the People room until they are face recognized again.
• You can create People tags for the existing files by using the same method as I suggested for Albums.  In other words, select the Imported Keyword Tag box so all of those files are filtered in the grid.  Create a new People tag for that person under the People tags.  Select all of the filtered photos (Ctrl+A) and apply the newly created People tag to those images.  A generic icon will appear in the People room for all of those newly tagged images.  If you do not want to face recognize those images again, wait for face recognition to be completed.  You can then select all of the UnNamed People stacks that have been created and click the Don't Show Again button.
• I don't really use Events and Places system tags.  So I'm not sure how those are treated when files with those tags are imported into a new catalog.  I suspect that they will be retained in some form or another.  It should not be too difficult to import a small sample of such files and see whether the result is something that is acceptable to you.  If I have some time, I will try a sample myself.  I am also going to ping @MichelBParis, another Community Expert, who may already know the answer for you.
• And my final point, which is probably the most important, is that when you import the tagged files into the new catalogue, make sure that you accept the option to import existing keyword tags.  A dialog will open up and, if you use the Advanced dialog, you can map the tags to an existing tag hierarchy.  My guess is that this may be useful in your situation because you will probably have overlapping tags in your 7 catalogues.

This is probably a lot to absorb, but once you start working your way through it, I expect it will become easier.  And as I said earlier, you always have the existing catalogue's to work with.  @MichelBParis will also have some advice for you if you want to use a single catalogue on both your desktop and laptop computers.  You are allowed to install Elements on both as long as you use only one at a time.  So, you may want to scan your files with your laptop, save the files to an external drive and import them into the new catalogue using your desktop computer.

[jsmcwilliam@gmail.com]

Thankyou Greg for your useful dialogue. I am working on it and following your guidlines building a new master catalogue. I will be in touch if I run into problems.

John