Known Participant

Question

Tools and tips for tidying messy file and catalog in PSE2020 on W10

Forum|Forum|2 years ago
April 16, 2023
3 replies
1501 views

I have used PSE for many years and am currently using PSE 2020 on a Windows 10 PC.

During migrations and in configuring OneDrive I have made a number of mistakes in how I have organised files and the PSE catalog and I am losing track of where files are stored - I haven't lost any files, yet, but I have come close to it and need to tidy things up before I do.

Currently, everything is in one catalog which has almost 24,000 items, including many duplicates. The files are spread over an inconsistent file structure which is spread over my C: drive, the old drive from a previous version of Windows (E:) and in the cloud in OneDrive. (This situation is too confusing to explain here, but it might help to give you and idea about how confused I am. The interaction with OneDrive is particularly confusing to me just now and this is where I am most concerned about messing up so badly I lose data. I have had to restore from local backup once. The restore got all the data back, as far as I can tell, but it further confused the folder structure. The restored catalog was relatively clean, but I think the old folders were in the PSE watchlist and the old folder structure has reappeared in the catalog with content being downloaded from OneDrive on demand. )

I think that the best thing for me to do is to grit my teeth and painstakingly go through all the folders that Organiser shows as having images in them, understand where they are and whether they are duplicates or not and create a new, clean catalog on a better planned folder structure by hand - by moving the files to a new, consistent location and importing them to a new catalog. (Then I can think about deleting the old files, but that will be risky.)

I want to be able to use the new structure from my PC and a laptop, so my current thinking is to move the files to new folders in OneDrive and create a new master catalog on the PC. I would then create a seperate catalog on the laptop which would watch the OneDrive folder for changes and keep in synch that way.

I expect this will be a very long and tedious and anxious process so and efficient and reliable process is key. Before I start, does anyone have any advice about tools, tips and techniques for manually cleaning up a messy catalog? For example, tools that would let me compare folders and highlight differences? Anything in PSE that reports on duplicate files, broken links etc? Is there anything in PSE that would restructure files, e.g. create a new catalog based on a consistent folder structure based on the date shot or which would copy files over to a consistant structure?

Windows

This topic has been closed for replies.

Greg_S.

Community Expert

@Billy5FE5, Michel has given you sound advice.

I am just curious whether you use keyword tags, face recognition and other Organizer features such as Albums. It may affect what approach to take in the future.

And while your main concern seems to be where your files are stored on your computer, the Organizer doesn't really care where they are stored unless, as in your case, you intend to use the same catalog on a desktop and laptop. Michel is an expert in that workflow.

When you get around to comparing different copies of folders, there are a number of utilities that can help you.

B

Billy5FE5Author

Known Participant

@Greg_S. Thanks for your interest in this.

I use tags a lot. Most of my images are tagged one way or another and I often rely on them to locate images. Facial recognition, yes I use that too, but I don't rely on it and I try to tag images by hand as I import them. Albums, I don't use much at all.

I take the point that Organiser doesn't care where files are stored. I am starting to understand that I may have been taking the wrong approach with the backup. I thought it would be best to use OneDrive for backup for everything I use (apart from PSE, I have Office 365 files, mostly, with an old iTunes library to complicate things) and wanted to include my photo library in the same approach.

Maybe, instead of having one approach to backup, I should rely on Organiser to backup the photo library and exclude the photos from the PC backup regime.

Above I said to @MichelBParis that I wanted to work on the folder structure rather than using tags and albums, but maybe using tags and albums is the best and safest first step on the way to fixing this mess. (Once it is sorted at that level, I could look to understand the underlying folders.)

MichelBParis

Legend

Hi @Billy5FE5 ,

There are a number of things to discuss in your situation. Before that, I would start immediately with two suggestions:

1 - Create a new organizer full backup on a dedicated external USB drive to save the current sate of your catalog and managed photo/video files. Don't work on deleting or organizing at that stage or playing with OneDrive.

2 - If I understand your project to share the final catalog and library between two computers, my strong advice is to choose to store both on an external hard drive (I have been doing this for years). That means restoring the above suggested backup on a new small SSD external drive. You first create the new master folder as the 'custom destination' to hold your photo folders tree as well as the restored catalog folder. You don't lose anything of your current "organization", you have a backup and you can even access the new catalog from two computers having the same PSE version by just plugging the HD alternatively on both. Use the option to keep the folder structure.

Already a few tips:

- when you create such a full backup what happens with the original folders structure?

OneDrive: the files synced with OneDrive are on a specific OneDrive folder, they will be backed up and restored with the same path as before, but that won't the real syncing system drive, just a 'normal' folder.

Different drives: in the restore process on a single drive, the files of a separate drive will be restored under a master folder named after the source drive.

Duplicates: Normally, the organizer should not contain duplicates in the catalog. In the organizer definition, they are totally similar in content and format, and are skipped at import or backup time. However, you can have visual duplicates, or files with the same name, bu not having the same date_taken AND file size in kilobytes. The backup you are doing scans the photos in your catalog, not on your drives. Duplicates on the drives are not backed up.

At that stage, we can start to discuss the cleaning and reshaping process of the catalog itself. Indeed, that may be a big task if we focus mainly in achieving a clean and meaningful folders structure. However, if we rather want to make the 'virtual' tag and albums structure better, allowing better searching and grouping of your photos, that should be easier and les dangerous.

What do you think about it?

B

Billy5FE5Author

Known Participant

Wow, two full, fast replies. Thanks to both.

@MichelBParis 1) Absolutely, I won't do anything risky without a full backup to an external hard drive. Fortunately (or sensibly) I had taken a full backup before my last mistake and was able to recover because of it, so the lesson has been learned.

2) I have read that the best way to access the contents of Organiser from two devices is to create a copy on an external drive and physically swap it over, so that probably is going to be the best way. I had hoped that if I used OneDrive I could find a better way of working that would:

a) give me a fully automatic, real-time backup process (I am lazy and need the backup to be as automatic as possible or I'lll start to skip it)

b) give access to images from other devices (like my phone) from anywhere. This universal access isn't important, though, it's just a nice to have.

I have to confess that I don't understand your advice about the backup and duplicates.

On the duplicates and not normally having them in the catalog, what I see on my catalog doesn't seem to be what you expect the catalog to be like. I have catalog entries that have the same name, size, date and time but point to seperate folders. For example, one entry points to E:\Users\BillyandJean\Pictures2020\2014 09 26\2014-04-19_14-17-03_999 and the other to C:\Users\billy\OneDrive\Pictures\2014 04 19\2014-04-19_14-17-03_999. When I look at them in file explorer, the files contain similar images and have the same size, date and time. The first is my original file structure but I honestly can't explain how the other one was created - it was probably part of my confused attempts to implement OneDrive.

If it is feasible, I would rather put the effort in to cleaning up the underlying folder structures rather than trying to solve the problem by tagging. I get it that tagging is safer and may take less effort, but this is just part of a general attempt to get the PC in order and I'd like to get this sorted at the underlying level. The question is whether or not it is feasible and worth the risk.

MichelBParis

Legend

2) I have read that the best way to access the contents of Organiser from two devices is to create a copy on an external drive and physically swap it over, so that probably is going to be the best way.

By @Billy5FE5

Doing a full backup and restore on a new external destination drive is the best solution. One big advantage is that it's the only 100% sure way to check if the transfer is ok. Note that in your case, you still have both the original catalog + data unchanged, plus a proven backup. You can start your re-organization job from the original computer disk or the external one, while keeping access to the previous situation.

I had hoped that if I used OneDrive I could find a better way of working that would:

a) give me a fully automatic, real-time backup process (I am lazy and need the backup to be as automatic as possible or I'lll start to skip it)

b) give access to images from other devices (like my phone) from anywhere. This universal access isn't important, though, it's just a nice to have.

By @Billy5FE5

OneDrive is indeed a good solution to backup your files, but not the relationship between the catalog and the media files. I like it to share some kinds of documents and a selection of photos with different devices when away from my computer.

Your experience with duplicates is not easy to troubleshoot. The 'filtering' of duplicates when importing (or backing up) works very well in most cases. For instance, it's the same when you use the 'watched folder' services or when you manually import from one of the watched folders. Same scanning and filtering the files in the catalog. However, the filtering algorithm has been said by Adobe to be more complex than the two criteria mentioned above: date_taken and file size. However I have never seen it described in detail. So, there are still situations when the filtering does not work. Fortunately, they are not frequent. Recently, that happened with importing from smartphones or cameras which stored duplicates for some reason. That happened with files imported from cloned disks; probaly also from a mix of original files with restored ones... Good luck to find the reason unless you have lots of duplicates for the same reason.

Ged_Traynor

Community Expert

Moved to the Photoshop Elements forum from Photoshop ecosystem

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded