Lots of dupe photo files - condense on disk and rebuild catalog?

[GaryWicke]

Greetings

It seems I have created a situation where I have quite a few duplicate photo files in different disk directories.  

If I combine all the photos into a single directory, letting the OS (Windows in my case) copy process help me eliminate the duplicate file names, can I then build a new Catalog and have LR Classic IMPORT them to the correct disk directories based on their capture date (YYYY-MM in my case) ?

If so:

• Do I need to create the disk directories ahead of time?
• If not, will LR create the directories it needs as it loads the pictures?
• What happens to all my existing thumbnails?
• Should I delete/remove my existing Catalog before starting?
• What else could go wrong with tis plan?  🙂

Thanks everyone for your help and suggestions!!

-gary

•  

[gary_sc]

Hi Gary,

When you say "Directories," I'm assuming you mean Catalogs? Please verify that.

One thing to watch out for is duplicate names with different images. This occurs when you've taken a whole lot of photos and you are then going on the same number but the 2nd or 3rd time around. One way around this is if you have also used folders for events to contain the images. That is one folder for the trip to the Grand Canyon and another folder for Aunt Maude's wedding. You get the idea. If a photo in each of those two events have the same name, they are probably not duplicates.

Next: DO NOT USE EXPLORER OR THE FINDER TO HELP YOU MOVE IMAGES THAT ARE IN LRC.

Sorry for writing so loud but the whole success of LRC is based on LRC knowing where files are. LRC doesn't really care for where they are so long as IT knows where they are. When you move things around in Explorer or the Finder, LRC doesn't know where they went to so you end up with thumbnails with exclamation points letting you know that LRC doesn't know where the original file is. It is not all that hard to reconnect them but it can be tedious and wastes your time.

So always move files and folders around inside the confines of LRC. ALWAYS!

As far as finding duplicates, here are some thoughts on this:

https://www.creative-photographer.com/find-duplicate-images-lightroom/

And you may wish to get some plugins for that. Here's a link that looks at this:

https://havecamerawilltravel.com/lightroom/find-duplicate-photos/

I hope I've given you thoughts to work with.

[GaryWicke]

Hi Gary_SC

Great name for a start!!  🙂

I understand the issues with using Windows Explorer in moving picture files around as you suggested.

I've created a bit of a monster for myself and not sure how it happened.

I know I have duplicate files of the same picture because I've examined a few.  I'm also sure, as you suggested, that I have duplicate file names storing different pictures.  I'm looking at a total of just over 7,000 picture files and due probably to my both inconsistent and uneducated methods of importing the pictures from both my iPhone and from my DSLR I have created a series of directories on my hard drive that looks like this:

2008

     2008-01

     2008-02

     .

     .

2008-01

2008-02

2009

     2009-01

     2009-01

     .

     .

2009-01

2009-02

So I have two (2) directories in many cases that represent the same timer period, say January 2008.

What I'm suggesting is to create a holding directory to store ALL the photo files with a separate directory for the duplicate names to be resolved later.  I then remove all the existing dated directories, delete the existing Catalog and IMPORT all the photo files back into LR letting LR create the directories according to their repsective 'Photo taken' dates taken form the file metadata.  This does assume I've set up the LR import process to use the desired YYYY-MM directory format.

I would then have to resolve the duplicate issues by maybe manually examining the pictures by name against the already loaded ones and import the ones deemed not to be a duplicate of an existing picture.

Will this work? 

I truly apprecaite any and all suggestions for better solutions!!

-gary

[dj_paige]

I wouldn't move photos or combine folders. There are utilities you can run in your operating system that searches folders and identifies photos that are duplicates, and that way you can delete the duplicates. I think there is also a Lightroom Classic plug-in that will do this.

[GaryWicke]

Hi dj_paige

Thanks for the feedback!

I really have two (2) issues: 1) Find and eliminate the duplicate photo files and 2) Re-organize my disk folders to a uniform YYYY-MM format.

To accomplish the 2nd issue I need to combine the multiple, usually only 2, folders I have that represent the same time period.  The example I used was January 2008 where I have one folder as c:\2008-01 and another as c:\2008 -> 2008-01.  I would like to migrate to the latter format where I have a top-level directory for 2008 and sub-directories under that for the individual months.

I will definitely look into your suggestions for finding the duplicate photo files, thanks!

Do you have any suggestions about how to move my files to condense them i nto the ultimate structure I want.

NOTE: Whoops, and it's a big one, I forgot to mention that currently the metadata 'Date taken' for many of the photos does NOT match the directory they're currently in.  So the re-organization I need is to get the files into the correct directory based on their 'Date taken'.

Thaks again for all your help!!  Much appreciated!!

-gary

[dj_paige]

2) Re-organize my disk folders to a uniform YYYY-MM format.

That can be a lot of work, and at least in the suggestions where you have to re-import the photos into your catalog, then you lose lots of information such as (not a complete list): edit history of steps, virtual copies, collections, pick flags, photo books, slide shows, web module.

A much easier way to do this is to forget about folders and use the Filter Bar, which can find photos by any date, even if they are in the "wrong" folder. This takes much much much less work on your part.

So my advice is to not re-organize your folders. Use the tools you already have, which saves time and effort, it seems like win-win to me.

[GaryWicke]

dj

The more I look at my files the more I think your suggestion is the best.  

Question: How can I get the 'Date' column in the 'Default Columns' filter to show in the YYYY-MM format rather than the YYYY-MM-DD format I'm seeing?

I've clicked on pretty much every Preference option but I don't recall seeing a Date format setting other then in the Import operation.

I've also started an effort to keyword all the photos for easier searching as well.

Thanks again for your great suggestions, hopefully you can resolve this latest one.

-gary

[dj_paige]

I'm not seeing YYYY-MM or YYYY-MM-DD in the Filter Bar. Where do you see this?

This is what I see

[GaryWicke]

Hi dj

Here is what I see:

It must get the breakout from the actual dates in the file metadata.

But even looking at the metadata of the pictures in a given directory it doesn't match where the filter criteria has put it above.

-gary

[Rob_Cullen]

Info to this situation-

The Date Column you see in the Metadata search can be changed in appearance by a [Right-Click] on the small icon at the end of the Column Header. The view can be Hierarchical or Flat.

[GaryWicke]

WobertC

Well I never knew that was there.  Thanks very much.  Always ready to learn the little tricks.

Follow up question.  Where does LrC get the Date breakdowns it shows in either display format?

Are these dates taken from the Catalog and the data captured each time a picture is Imported?

Is it possible to change the display level to just YYYY-MM?

Thanks for all the help!!!

-gary

[Rob_Cullen]

-Metadata is read from the camear file when it is imported.

-Changing the metadata in LrC records the changes in the Catalog.

-Any changes in LrC can be 'written' into the file by a 'Save Metadata to file'.  eg. Change Capture dates in LrC and write the change to files [Ctrl+S].

change the display level to just YYYY-MM?

Just click on the 'MM' spinner triangle-  

    

Note that the 'parent' MM or YYYY always shows the total number of files. (What else could it show !!)

[Theresa J]

LrC will create new folder directories by date IF you import using the Copy method. You could choose a new drive to copy the images to during the import. This will organize the files by capture date. After everything is imported and organized, you could then delete the photos from the original location.

[GaryWicke]

Hi Theresa

Thanks very much for your suggestions!  I had forgotten about using the 'Copy' option, big mistake on my part, thanks!!

I also like your idea of putting the files on a different drive to keep them separated from the new target directories.  That gives LrC a better chance of creating the directories I want.

Thanks very nuch!!!

-gary