P: Moving files, resulted in deleting the files

[jkborys]

I am using Mac OSX and LR CC 2015.

I have a workflow that involves importing thousand of raw photos and then sorting them. In order to sort them, I take a group of photos of the same subject and move them to a temp folder and then repeat until all the photos have been moved to sub categories based on there names. I then move the photos out of the temp folder into the original folder and delete the temp folder.
While attempting to move the folders containing the photos, one of the folders and the 97 photos under it was lost. There is also a descrepancy in the number of photos copied 97 vs 76. And I can no longer find the photos on my hard drive. i did NOT delete the photos. LR seems to have deleted them. This is a HUGE problem. KimBenchProblem_inline-81a5d584-d717-4e35-ac1b-a8c550acdee7-631396214.jpg

[Jbean25]

Today I moved a folder inside LR to an external HD. The folder itself was over 40GB. The progress bar stopped about 1/3 way through the process, and after 15 minutes no more files were being moved. I finally stopped the move by clicking the "x." When I looked at the size of the folder in the external HD and the size of the original folder still remaining, I had lost over 25GB!!  
Arguably the most important function LR has is not image organization, it is image preservation. And if files just disappear due to LR glitches, then steps need to be taken to fix that as priority #1! 
I will not be moving files within LR any longer, and I will not recommend that anyone do so. If I know of someone who is considering using LR, I will suggest they look at alternatives.  Unbelievable!

[John R Ellis]

Many of us long-time users make the same recommendation: If you're moving large number of photos and folders, do it Mac Finder or Windows File Explorer.  Then relink the missing folders in LR.  There have been infrequent but persistent reports about disastrous data loss for many years.

[FeedbackCommunityMember]

There is something really different in the way LR and Elements Organizer deal with moving files and folders.
With Elements (at least not too old versions) if you get a problem like a power outage when moving files, what happens?
- The original files are not deleted, they are still there.
- The part of moved folders is on the destination master folder or drive. The catalog rolls back to the original situation. You only have to delete the partial copy and restart the process.
Seems like there is a two steps process : a copy followed by a delete only if the first transaction has succeeded.

In the Organizer or LR, it's obvious a backup is in order before such a risky move.

On the other hand, relinking in LR is much better than in the Organizer, so, in the Organizer I am often tempted to recommend moving by drag and drop from the folders panel.

I have not met the issue with moving files in LR (infrequent reports...) but I wonder what happens in this case with the original files (can they be recovered from waste bin) and the original catalog (which may have been backed up) ?

[Todd Shaner]

"I will not be moving files within LR any longer, and I will not recommend that anyone do so. If I know of someone who is considering using LR, I will suggest they look at alternatives.  Unbelievable!"

It's all in a word and using a "Move" operation in ANY application with a large number of files & folders is dangerous. Any number of things can go wrong during the transfer such as a power outage, memory issues, drive issues, etc. that can cause files to be lost.

Instead use 'Copy' and 'Paste' to add a copy of the files and folders to the new drive or folder location. Once completed you can compare the original location top-level folder for Size, File count, and folder count and compare it to the new location copied to. They should be identical. If not you still have all of the files at their original location and can try again.

Scroll down to reply #13 at the below link for more details:

https://forums.adobe.com/message/6590881#6590881

It's always recommended to backup the LR catalog and image files before doing ANY operation on a large number of files, but the above Copy, Paste & Compare procedure is pretty safe.

[Jbean25]

Is this copy and paste method what Adobe officially recommends? Or are they on record recommending that people move files within LR? If it's the latter then they should at minimum issue an official policy that warns against major unrecoverable data loss during this procedure. 

I see in the site you referenced someone suggests changing the name of the original folder to break the LR link and then copying and pasting. I will probably do that. 
Regardless of what my happen to cause files to be lost, this was not due to a power outage. This is a LR issue and has been documented in the past-- of which this thread is evidence. I do back up the LR catalog frequently. That is a moot point. 
I did have most of the files backed up, thankfully. However, now I have the remaining files left on a HD, which LR did not move, and which are contained in collections, and a backup in a separate location which does not contain every file, but most, that LR has no awareness of. So it's a complete mess. Thanks Adobe!

[Todd Shaner]

"Is this copy and paste method what Adobe officially recommends? "

To the best of my knowledge NO and when moving a small number of files there generally is NO problem. The problem occurs when moving a LARGE NUMBER of FILES for all of the reasons I stated at the link provided.

"I see in the link someone suggests changing the name of the original folder to break the LR link and then copying and pasting. I will probably do that."

No need to do that! When you copy the folder structure to a new drive LR is completely unaware of the operation!

"Regardless of what my happen to cause files to be lost, this was not due to a power outage. This is a LR issue and has been documented in the past. I do back up the LR catalog frequently. That is a moot point."

So all the more reason for using Copy & Paste from outside of LR, agreed?

"I did have most of the files backed up, thankfully. However, now I have the remaining files left on a HD, which LR did not move, and which are contained in collections, and a backup in a separate location which does not contain every file, but most, that LR has no awareness of. So it's a complete mess."

 I totally agree and it's a mess that could have been prevented using Copy, Paste, & Compare from outside of LR! To "repair" the damage you can try using File Synchronization software to find and copy-over files in the original folder and/or backup that are missing at the new drive location.

https://en.wikipedia.org/wiki/Comparison_of_file_synchronization_software

I use FreeFileSync available here: https://sourceforge.net/projects/freefilesync/

Make sure to use the 'Donate' link and not the 'Download' link, which contains adware. There are also numerous other synchronization apps as shown at the above Wikipedia link.

[Jbean25]

Todd, so you agree that this is 1) An Adobe LR issue and 2) LR official documentation online makes no mention of it and in fact advises users of LR to engage in a risky practice that could result in major file loss and 3) LR has said this issue is solved (according to this thread) when in fact it is not solved. So among people in the know or who frequent this forum, the copy/paste method is known to be safer. Well that's great, but that is not enough, not by a long shot, wouldn't you agree? Where is Adobe on this issue? 
For some reason you seem to be doing everything except laying blame at the feet of Adobe. Why is that?

[Jbean25]

Todd, so you agree that this is 1) An Adobe LR issue and 2) LR official documentation online makes no mention of it and in fact advises users of LR to engage in a risky practice that could result in major file loss and 3) LR has said this issue is solved (according to this thread) when in fact it is not solved. So among people in the know or who frequent this forum, the copy/paste method is known to be safer. Well that's great, but that is not enough, not by a long shot, wouldn't you agree? Where is Adobe on this issue? 
For some reason you seem to be doing everything except laying blame at the feet of Adobe. Why is that?

[Todd Shaner]

"Todd, so you agree that this is 1) An Adobe LR issue"

Yes,and I feel your pain!

"2) LR official documentation online makes no mention of it and in fact advises users of LR to engage in a risky practice that could result in major file loss"

That is correct. Users in fact are advised NOT to move files and folders from outside LR since they will become "Missing" and need to be located and reconnected inside the current catalog. It's a "damned if you do and damned if you don't" situation.

"3) LR has said this issue is solved (according to this thread) when in fact it is not solved."

Correct.

"Where is Adobe on this issue?"

Adobe is aware of the issue and made at least one attempt to correct it with the LR CC 2015.12 bug fix. This report was entered over three years ago, but to date has received only 10 'Me To' votes. That's not a huge number of people claiming to have experienced the issue. That said there have been numerous reports of this issue in the LR forum, but Adobe Engineering rarely views those posts. Example: https://forums.adobe.com/message/8240503#8240503

"For some reason you seem to be doing everything except laying blame at the feet of Adobe. Why is that?"

The primary reason is because I simply don't trust LR or any other application to move large numbers files "automatically" on my desktop system. If I was using a server-grade system with ECC Memory, RAID 5 storage, and a UPS perhaps I would be more trusting. To be honest I do occasionally use LR when I need to move a few files, but in general once files are imported into LR I don't move them. Instead I use keywords and collections to "organize" my image files.

Even if Adobe implemented file & folder moves using an "automated" Copy, Paste & Compare checksum routine there's no guarantee that files won't be lost due to external system issues. That should make it more reliable, but the only 100% foolproof method is using Copy, Paste and a human comparing the origin versus destination results. It's a relatively simple & painless operation, especially if you're moving one top-level folder containing thousands of file. This is the way I moved files & folders for years before LR was even available so maybe I'm old-school.

[Rikk Flohr: Photography]

Just an update: I've rerun John's test shown above that originated this bug against 6.9 and it shows that this particular issue remains fixed. It is likely the two recently reported issues on the other thread are not exactly the same issue as was reported here.

[John R Ellis]

The link above is broken; the topic Rikk is referring to:
https://feedback.photoshop.com/photoshop_family/topics/lightroom-deletes-files-when-moving-folders

[FeedbackCommunityMember]

I have came across this exact problem last night. I'm using MAC OS Catalina 10.15 and LR Classic. 

The files were imported from SD card into my internal HD. I then dragged and dropped the folder in LR so the folders were moved from internal HD to external HD. LR started transferring the files and completed without showing any error. However, the actual files were not moving to the new folder and the files were gone from the original folder. I can see the folders showing on on LR with preview images but there are no files. In the external HD were those folders are located, those folders are showing as 0 bytes. The original folders in the IHD are also gone.

I replicated the process again and kept my eyes on the OS finder window (where the original folders were stored) during the transfer. When the process was completed in LR, THAT folder are gone and there are no files in the new folder.

[Rikk Flohr: Photography]

Pearl, I also saw your post on the LIghtroomForums.Net.  I forwarded your post to engineering and they have some information they would like to collect from you regarding this serious problem. I will contact you via email with instructions.