Importing files from a new external hard drive

[Russ001]

my old external 3TB drive was full so I purchased an 8TB replacement.  I copied the files as is from the old drive to the new one.  I had removed all of my prior LR catalogs so I was starting from scratch (that may not have been the right approach but was what occurred).  I have roughly 30000 images including all the side files making the 3TB of image data.  My files are organized such that I have one primary folder called “Lightroom” and all the images are under this one as sub- folders as well as sub-sub-folders etc.  I then clicked the import tab, highlighted the high level “Lightroom” folder, marked the “Add” tab and the files started loading.  When it completed, several folders and sub- folders were omitted.   Of the 30000 images, only about 23000 showed up.

[Bill Sprague]

Did you delete the catalogs and their backups?  Do you still have the old drive?

[Russ001]

Yes, I had deleted all the catalogs within LR to start over new.  I realize now from other comments that there was no need to do that.  I still have the EHD original data.

[ManiacJoe]

Assuming that you mean that the import process skipped some folders that existed on the hard drive, if you do a second import, do the previously skipped files get found and imported? (Turn on the skip duplicates option.)

Yes, you are correct in that deleting the old catalogs and doing a new import was the wrong approach. The correct thing to do was to open the old catalog and "relocate" the "lightroom" folder to be on the new drive.

[dj_paige]

I copied the files as is from the old drive to the new one.  I had removed all of my prior LR catalogs so I was starting from scratch (that may not have been the right approach but was what occurred).

Absolutely the wrong approach. No need to import anything. Simply point Lightroom to the files' new location, explained in this document: Adobe Lightroom - Find moved or missing files and folders

[JoeKostoss]

Russ001  wrote

my old external 3TB drive was full so I purchased an 8TB replacement.  I copied the files as is from the old drive to the new one...

I see this over and over again.  When an EHD gets full, there is absolutely no need to put all those files on a new larger EHD.  Lightroom works perfectly well with several EHDs.  I recommend keeping the files on the first EHD and start adding new files to a new (and larger if you want} EHD.  Keep both EHDs plugged in until such time that you find yourself not referring to old files and concentrating mostly on the new.  You can then put the first EHD in storage.  Pull it out only when you need it.  Lightroom will remember!

[Bill Sprague]

I agree with JoeKostoss's approach.  

There is another approach for users that want to keep it all on one EHD.  Plug in both the new and old drives to your computer.  Using the operating system (Finder or Explorer), copy all the files from old (like "D:\") to new (like "F:\") so that the only thing different is the drive letter.  The file structure has to be exactly the same.  Unplug and store the old drive.  Rename the new drive to what the old was named.

[Bill Sprague]

This won't help Russ (the OP) much, but for any other readers, there is another consideration. 

External hard drives keep getting cheaper.  $100 will get you 6TB!  Buy them in pairs.  Use the second one to backup the first with software like SyncToy,  GoodSync or other. 

Part of this includes setting Lightroom Classic CC to write the catalog backups to the primary external drive.

If you value your work, you will buy three, alternate the backups and keep one away from wherever you do your work.

[Russ001]

Yes, this is what I should have done.  The missing folders issue was a strange one to me as I started out with a clean drive.  I noted that when I did the copy of files from the old to the new drive, that the ‘don’t import duplicates’ was checked that should not have made an impact when copying to an empty drive.  Anyway, I cleaned the new drive and did the copy over again with that not checked and everything transferred fine.  So, I am all set now.  One thing I need to start now is to go back an delete many of the images that I have no plans to use...like many bracketed images...yikes!   

thanks for for all the helpful comments all!

[TheDigitalDog]

JoeKostoss  wrote

Russ001   wrote

my old external 3TB drive was full so I purchased an 8TB replacement.  I copied the files as is from the old drive to the new one...

I see this over and over again.  When an EHD gets full, there is absolutely no need to put all those files on a new larger EHD.

There's some very good reasons to do so. One is having all the data in one location (drive), allowing the user to clone (backup/copy) to other drives in one operation. Or moving that external drive to another machine running LR where again, all the images reside. Plus one should seriously consider saving the catalog, the previews and the resets (Store Preset with catalog option) on that one big drive. Scattering this data over multiple drives when one big one will hold all the data serves no purpose and simply confuses the entire issue of finding and using said files.

[JoeKostoss]

thedigitaldog  wrote

There's some very good reasons to do so...

I agree that there are some good reasons...for some images...but for 3TB worth of images...really?  To me, that doesn't seem realistic.  In this case, we are looking at 2 EHDs, not multiple EHDs.  To me, carrying 2 EHDs to another computer isn't a deal breaker.  If some images need to be moved to the new EHD, so be it, that can be done as needed.  Moving 3TB to the new EHD makes the available space on that new EHD only 5TB, 62.5% of its full capacity (Looking at it another way, that's a 60% increase in the cost of new storage space, pay for 8TB, get 5).  OK, so for a few months, or maybe a year, this might be a little inconvenient, but in a year or so, the old images on the old EHD become sooo last century, and life goes on with the present and the future.  What happens when the new 8TB (5TB really) becomes full?  Buy a 20TB (12TB really) and copy all images and continue?  So, yes there are some good reasons, but I don't think copying all images to the new EHD is worth the problems that this OP (and others in the past in this forum) are going through.  For me, the best choice is still to power through the transition...and move on.

[TheDigitalDog]

JoeKostoss  wrote

When an EHD gets full, there is absolutely no need to put all those files on a new larger EHD.

I agree that there are some good reasons...for some images...but for 3TB worth of images...really? 

Yes, really and PLEASE make up your minds! I have, as to why it makes so much sense to dedicate a big drive for all images. There are more but I think I've provided enough.

[Russ001]

Would saving the sidecar files on multiple locations be wise to do?  I currently save backups to my laptop internal drive.  Seems making a copy to the external drive would be wise.  if one is corrupted, can LR be pointed to the other backup?

[JoeKostoss]

Save not only the sidecar files but the raw and image files as well.  Save them so that the second EHD is an exact clone of the original, including the folder structure.  That way it becomes a very easy painless process to point LR to the backup.  On my Mac, I use SuperDuper by Shirt Pocket Software to auto clone every evening so that the 2 EHDs are always exact clones.  SuperDuper is only for the Mac, but there are other apps that will do the job on a PC.

[Bill Sprague]

Wherever you have a backup copy of a RAW file that has an XMP sidecar file, that file should be with it.  Lightroom does not normally write sidecar files, but you can turn that option on.  

Other than writing them as an option, Lightroom doesn't normally use sidecar files.  An exception might be if you were to send a RAW file that you "Developed" to someone or some other computer for more "Develop" work.  You can pass on the work you've done in the sidecar.  The receiving person or computer could "Import" the RAW file.  At first it will look untouched until you have Lightroom "Read Metadata from File".  That causes the XMP data to be written into the catalog and the Develop sliders will move. 

XMP sidecar files can also be used as a sort of last ditch backup for Lightroom Develop module work.  Normally, Lightroom ignores XMP files because it uses the catalog.  If the catalog ceased to exist, Develop work is lost.  If the optional XMP files are present, a user could Import files into a fresh catalog, "Read the Metadata from File(s)" and that would put the Develop work into the fresh catalog.    

In your situation Russ, if you have XMP files, you could be creating a new catalog that contains your Develop module work.