1 Correct answer
With total respect for your difficult situation I believe that Lightroom is not for you. It is software that is different to any other and must be understood to work at its best.
If you are not willing to re-learn how Lightroom 'really' works then perhaps 'ACDSee Photo Studio' is more appropriate for you to edit your shots. Good luck in your future photo exploits.
I repeat my suggestions for learning:
Free eBook from Victoria Bampton for Lightroom-CLASSIC: https://www.lightroomqueen.com/blog/
Also
...
55
Replies
So the simplest approach for you is to-
Select the topmost folder in the Folder panel, [Right-Click], and [Synchronize Folder...]
Check every box in the Sync Dialog and [Synchronize]
You will get the 72,000 photos to add to the catalog, 'Missing' photos will be corrected. Your Catalog will reference every photo.
This procedure will take a long long time!
You will then see multiple same photos in the Library Grid, but that is to be expected (especially if you have "Show Photos in Sub-folders" checked in preferences.)
You will lose editing history on 'missing' photos but that is the least of your problems.
You will not have editing history for photos you edited in 'old' catalogs, but also you seem not concerned with this as you have your 'Developed' exports.
Then- For future exported photos you must check the [Add to catalog] option in the Export dialog.
Finally I ask- Why are you exporting EVERY edited photo to a sub-folder?
The Catalog 'saves' every edit in History and a 'Developed' (by your definition) photo can be exported at any time from the Catalog when needed for a specific purpose. It is rarely necessary to export derivative photos to sub-folders as you have been doing.
And- Never again look at your photo files in File Explorer or a File Browser.!
Started the process. Went ahead and found almost all of the missing files, did the "save metadata to files" process, and started Sync on Pictures. Quickly realized my Game folder, with 250G (80,000 pics), was included, so I stopped it and restarted on "My LR Photos," which is where "all" my pics are, or were before I started working on the scanned slides. Didn't realize/forgot I'd been working outside the "My LR pics" folder.
When that process concluded and I realized it hadn't reached the scanned slides (nor a lot of other folders), I moved the game pic folder out of pictures intending to sync at pictures.
Had to make that move outside of LR because now it was grayed out in LR. (Along with everything else previously showing in LR but not in "my LR photos" and not a parent folder to that!!) No matter, they're game camera, trail camera pics. Not for LR.
Then I realized that even though "pictures," as a parent folder to "my lr pics," was not grayed out, it cannot be selected and I cannot do a sync on it.
I had no clue that syncing a folder would result in every folder previously in LR but outside that folder would now go gray, outside. When I did the trial run on one smaller subfolder that did not happen. I do not see why it ever should. Seems quite perverse.
So now I can't sync from "pictures" and everything outside of the "my lr pics" folder is grayed out. Something may occur to me overnight, but right now I can only see importing (re-importing!!) all those folders to get access to run Sync.
What happened? What now?
Here's the picture:
Stop moving folders 'outside' of Lightroom! You are creating havoc.
Your screen-clip is waiting to Sync 60791 photos- Have you clicked on the [Import] button yet?
WobertC wrote
Stop moving folders 'outside' of Lightroom! You are creating havoc.
-Understood, but: What doesn't appear in LR, I can't move in LR The game cam pics haven't been imported because there's no need. Actually I need to trash most of them.
Your screen-clip is waiting to Sync 60791 photos- Have you clicked on the [Import] button yet?
-I am not at that computer at this moment, but I believe the Sync is complete. It appeared to be. If I open "my LR pics" everything that should be there is there, with only one or two odd empty folders. The folder tree looks 100% nominal and the contents of the folders looks 99+% nominal. Here it is:
It appears the Sync worked. My issue now is that the Sync process had unexpected impact outside of the target folder.
After the sync every other folder that had been in LR ("Scanned Slides," for instance) is now out of LR, grayed out, with only parent folders to the synced folder ("pictures") visible but dead, unclickable. Can't sync 'em.
So, why did this happen, and how to sync the now ejected folders within pictures? (It's neither here nor there, but when I did the test sync on one sub-folder this ejection did not happen.)
I could re-import them, but that seems unnecessarily roundabout and somewhat fraught, since the originals, having already been imported once, are now grayed out (I looked, some are grayed out, I presume those are the originals) and LR will not allow them to be imported. It will tell me they're already imported. Well, that's what I recalled, but actually LR just won't allow the grayed out files to be selected. Regardless, they can't be imported.
Maybe go back to a previous catalog backup, I'm not sure what I'll find there or how to get there. Neglected to do a backup before the sync so at a minimum I'll have to find all the lost pics again. What a tar baby.
It's beginning to seem I'm a hopeless case. I've been digging in my files, comparing what's showing in LR to what's showing in a complete view (using ACDSee) and I have two full pages of notes. Nothing really adds up.
One thing I noticed first thing - The grayed out folders I was so concerned about aren't. You can see I got myself into the Import screen and there they are grayed out. Hopeless.
That means one thing I can do is run another Sync from the Pictures folder. I may start that in a bit.
Studying the differences in the Synced portion of LR and the complete file structure there is a pattern - with bothersome exceptions! - that if the folder consists of JPGs only (no DNGs or RAWs) it will not appear in the synced LR. There are quite a few of those.The few exceptions sometimes consist of badged files, sometimes not.
Something I have yet to see an exception to is that if the folder consists of both JPGs and DNGs (or RAWs) it will appear in LR, with the appropriate number of files (half as many as are in the doubled up JPG + DNG file).
Something to do with the "add to catalog" choice in export, maybe. The parent folder gets JPG and DNG if checked.
Some screenshots.
ACDSee above, the same area in LR below. Note that every folder called or marked LR is JPEG only, and there are lots of folders absent in LR.
And here's a set showing an exception to the all JPG folder not showing up in LR:
Adding to my #49:
Haven't done the sync on pictures yet. Wanted to run save metadata to files again first. On 1,294 files could not do this because of "read only access."
All these were from scanned slides done 9/15 to 9/25/18. All have been edited and show badges. Can't be read-only?
Cannot see how to change that permission. Read that sometimes LR says shows that error when in fact the metada has been saved, but found metadata date and it is the date scanned, back in 2018.
How can I write the metadata to these files?
In reading about this I decided I decided to check (mark) "Automatically write changes into XMP." Is this OK?
Separately, maybe I'm nuts but I would have sworn that LR was recently - after the sync I'm not sure even happened now - showing all my subfolders of scanned slides. There are three such folders directly under "pictures." Now only one is showing.
There are actually a total of 13 folders directly under "pictures." (Well, "pictures>pictures." Don't ask.)
Here are the views:
Something I have yet to see an exception to is that if the folder consists of both JPGs and DNGs (or RAWs) it will appear in LR, with the appropriate number of files (half as many as are in the doubled up JPG + DNG file).
Set the option in Preferences / General tab- [Treat JPEG files next to raw files as separate photos.]
If this is not checked then only the DNG (or RAWs) from a RAW+JPG 'pair' will import to the Catalog.
After checking this option you can Folder Synchronize to import the JPGs that were not showing in the catalog. Check it 'on' now!
Wanted to run save metadata to files again first. On 1,294 files could not do this because of "read only access."
If this is your master working catalog- Don't bother! Worry about this later- and turn 'off' "Automatically write changes into XMP." Get you Catalog sorted first.
Your Catalog only contains 7824 photos as yet, so you have a long way to go to see all your photos in Lightroom! You will never see the same in ACDSee until you have Synced (or Imported) every file into Lightroom!
After your #51-
Didn't understand that about:
Set the option in Preferences / General tab- [Treat JPEG files next to raw files as separate photos.]
If this is not checked then only the DNG (or RAWs) from a RAW+JPG 'pair' will import to the Catalog.
I was already seeing the doubling, JPG + DNG. Must be LR terminology issue.
[From here might save yourself some time and aggravation by skipping to the bottom.]
Anyway, I did it, and the sync did not take long. Looked like all the folders were there. Saw no DNGs, only JPGs. Folders with my "LR' subfolders seemed to both (parent and sub) be edited, identical. Same folder outside of LR - some of them anyway - were showing originals (original originals) in the parent and edited in the sub. I was exploring it and about to try to comment on it - some few folders were empty - when I noticed the picture count was not showing.
Tried clicking on the first (top) "picture" folder and couldn't. Fiddled a bit and suddenly I was back to the pre-scan view! I decided to see what doing another scan would show and after the count I saw it was about to import almost 90m pics. Like I hadn't just done an import. Here it is, and this is AFTER an apparently successful import:
If this is not all nonsense I don't know what it is. Hit sync (again) and in less than one minute I have this:
Just like the first time - same screen I remember, no picture count.
Maybe I see now, and if I'm right it's yet another case of LR terminology disease.
Sync is just a name for universal import! When you "Sync" you just get a count, nothing else. Progress bar goes all the way, "finishes," 
but NOTHING IS DONE! All the "Sync" operation does it set up an IMPORT. Unless and until you Sync + Import you've done nothing.
Up to here all indications were that Sync was do some kind of import. Sync alone does nothing that I can see. That's would explain why I was back at square one after doing it.
Oh, well. I'll hit Import.
Must be LR terminology issue.
And have you read the free eBook cover to cover yet? https://www.lightroomqueen.com/shop/
Hit sync (again) and in less than one minute I have this:
Yes, and Lightroom is still searching for files that are not in the Catalog- Your 'Sync' screen-clip is up to 8575 and Finding.... You have many more!!!
Unless and until you Sync + Import you've done nothing.
Correct! Yes- You have to hit the [Import] button. The Sync screen is only a Preview of the files it will Import when you click [Import].
And you still show only 7824 photos in the Catalog.
The Progress Bar will usually pause (about halfway) until you click the [Import] button to allow the Sync to continue.
Sync Import seems to have hung up after about 45 minutes. This morning it was still where it was last night, at some 80%, showing some videos it can't decipher. I hover/click on them and get the wait circle and eventually the exclamation point and the message, as below. It took a while but they're all marked now, as you can see, but the progress bar hasn't moved. Most of the videos look like nonsense out of my wife's camera - anyway, I'm not concerned about them. Intending to x out of the process hoping all will not be lost. It isn't going to finish, and I know nothing else to do.
As far as references, I printed the one you suggest and skimmed it. I have several much more detailed books on LR and have read much. Nothing much seems to speak to my issues. The little LQ download doesn't treat Sync, and elsewhere she warns users to run quickly from anyone suggesting sync. [No worries, you're with her] From her 7/2 email:
"If someone tells you to synchronize a folder, run away even faster. Although it can be a useful command, it's also the most misunderstood, and can have catastrophic results if misused."
I've googled the hung up sync issue and while it is common most hits have to do with syncing to the cloud or between Classic and mobile. Some pissed folks, but nothing that applies here. Suggestions I see are pause and then . . . well, I don't see pause. Googled, go to Identity plate , went there, nothing. Click on" photos syncing" and delete them (that's a solution?) . . . no such button.
Went back to LR to see if I could delete the problem files and, much to my surprise, found LR still working on syncing videos. There were 11, per the screenshot above, and now there are 26, and most of the exclamations marks have disappeared. Hovering over the menu indicator brings up "working . . ." instead of the error message in the shot above. Update: Now there are 28 videos shwing and none with exclamation marks. If all are "Loading" this will never end?
Hoping to move things along, and assuming it would do no harm, I removed all 28 videos from LR. It's still hung up. Doing nothing I can see, maybe wrestling with the next video.
What can I do to save my work and stop this?
________________________________
The import so far looks fairly nominal. Most parent folders have two of each file, the original and one edited. (I think this is the only case where I see obvious originals.) Subfolders have the edited file. None have badges. So, the original and two edited. (Mysterious to me the reasons why, the purpose for the duplication. Understood the parent folder is "showing the subfolder," which in LR speak means showing the files contained in the subfolder. The point . . . lost on me.)
An exception to the pattern of if there's a LR subfolder, there's two of everything in the parent folder, is every file in "scan staging groups - temp." In that group every parent has one of each file, identical to subfolder.
No subfolder (LR) means one of each file in (what would be) the "parent."
The problem metadata files show exclamation marks "error writing metadata." Where there's a subfolder, half of the files in the parent show badges. None appear to be originals, and there are two of most. I can't really find a clear pattern but badges and exclamation marks seem to go together. Files in "LR" subfolders have no badges or exclamation marks (no metadata error).
Found three cases (in "scanned slides") of missing subfolders. All the parent folder files had badges and, with the exception of one file in one folder, the metadata error mark as well.
I seem like I am going down a rabbit hole here, but I hope I am shedding some light on the topics!
So to answer some specifics in you last post (we are crossing posts here
)
I don't think the unchecked "add to cat." box has anything to do with what puzzles me.
It does!- It will not show in the catalog library unless you do "add to cat" (or import)
I know that what was the parent folder will no longer show up in LR after the export.
WRONG! It DOES SHOW- It always remains in the library (Unless you delete it!) It is the 'Exported' folder that does not show UNLESS it is Imported by some means.
Since I created the "1971 GAD" folder in LR, I'm puzzled why, after the export, it was grayed out.
Yes you might have created it (which was not necessary), but you have not āImportedā the exported photos to the folder in Lightroom so Lr continues to show the 1971 folder (that you created) as empty (greyed). The Export process can do BOTH- Create the 1971GAD folder AND show it in the library.
But I'm much more puzzled why LR exported to GAD when "1971 GAD" was specified.
I think you are still looking at your original files in the GAD folder with the Develop edits showing on the images.
If LR somehow could not find the 1971 folder, shouldn't it have said so when I tried to export to it?
Maybe it did, but you just cannot see the exports in Lr as 'add to catalog' was not checked!
How did the files wind up in GAD?
Are you still looking at the Developed originals in the GAD folder imported to the catalog?
If that's not curious enough for you, viewing the folders outside of LR I can see them in the 1971 folder. LR put them there, and then lost the folder?
No it did not lose the folder, it just does not SEE the folder UNTIL you "add to catalog" or Import the folder of exported images.
I had to delete the 1971 version - I think maybe I had to go outside of LR to do it - and rename GAD as 1971 GAD.
Do NOT delete or rename anything! you will just confuse the Lr catalog even more and have more 'missing' files. ('Missing' as in the sense that Lr has lost its links to the files on the hard-drive.)
The "Rabbit Hole" gets deeper
Can I demonstrate my thoughts on workflow to you with screen-clips? Try following this procedure as a test.
1) I have an image from the camera that I have imported into Lightroom. It is called {Original Camera File.DNG}
This file is located in a folder called {IMAGE ORIGINALS}
2) I do some Develop edits in the Lightroom Develop module. I invert the Tone Curve (easy to see)
Note that now this thumbnail shows the 'Developed' BADGE. It is still the same {Original Camera File.dng} It just 'looks' visibly different in the Library, Loupe and Develop views of Lightroom.
3) I EXPORT this image to a sub-folder called "LR" (as you are trying to do) with these settings-
I have NOT created a {LR} folder prior to opening the Export dialog- I will let Lightroom create it for me.
The [Add to This Catalog] option must be checked if you want the folder and exported image to appear in the catalog!
4) The result is this- The sub-folder {LR} has been created automatically and it contains the EXPORTED JPG image.
Note: This thumbnail does NOT SHOW the Developed BADGE! It is a brand-spanking-new JPG image file!
5) Now- If i have the Preference set 'ON' to "Show photos in sub-folders" -I see this when I select the parent [IMAGE ORIGINAL] folder-
The .DNG with the 'edited' badge is the "Original".
The .JPG without an 'edited' badge is the Exported file.
I cannot seem to explain any better the process, but I can see the problems you are having and the possible errors causing these problems, including-
Multiple catalogs, Multiple versions of Lightroom, Not adding or importing ALL images to one catalog, 'Copying' at Import (as discussed by dj_paige, etc.
Time for a coffee
WobertC wrote
The "Rabbit Hole" gets deeper
Can I demonstrate my thoughts on workflow to you with screen-clips? Try following this procedure as a test.
1) I have an image from the camera that I have imported into Lightroom. It is called {Original Camera File.DNG}
This file is located in a folder called {IMAGE ORIGINALS}
2) I do some Develop edits in the Lightroom Develop module. I invert the Tone Curve (easy to see)
Note that now this thumbnail shows the 'Developed' BADGE. It is still the same {Original Camera File.dng} It just 'looks' visibly different in the Library, Loupe and Develop views of Lightroom.
Here we differ, but it's "only" a matter of semantics. To me, it's developed, it looks different, it is different, it's not the original. What's the same is the original, and that's kept outside of LR. We're changing it in LR, that's what we're about. Original=unchanged.
So, we're not in Kansas any more. I'll leave it, just remembering that semantics issues can cause understanding issues.
3) I EXPORT this image to a sub-folder called "LR" (as you are trying to do) with these settings-
I have NOT created a {LR} folder prior to opening the Export dialog- I will let Lightroom create it for me.
I might be missing something. About letting LR create the folder, there's already an original folder, you're exporting to it. All LR is creating is the "LR" subfolder, and you're telling it to, and naming it. That's my routine.
The [Add to This Catalog] option must be checked if you want the folder and exported image to appear in the catalog!
4) The result is this- The sub-folder {LR} has been created automatically and it contains the EXPORTED JPG image.
Note: This thumbnail does NOT SHOW the Developed BADGE! It is a brand-spanking-new JPG image file!
5) Now- If i have the Preference set 'ON' to "Show photos in sub-folders" -I see this when I select the parent [IMAGE ORIGINAL] folder-
The .DNG with the 'edited' badge is the "Original".
The .JPG without an 'edited' badge is the Exported file.
That's a great explanation, and I know it didn't come easy.
Off topic alert: I went to LR to try and replicate what I recall, and it is in "Navigator," where I was doing a search per instructions from djpaige. It won't open my D drive, where all the folders are. I'll close and reopen, but . . .. OK now.
With "add" checked, that's what I get, too. I've never paid any attention to it since it's about "catalog" which I prefer not to deal with. Looks like a real rabbit hole to me. My catalog is the file tree. I'm a no catalog guy and one catalog = no catalog, fine with me. I don't add descriptions, etc., to my photos. A catalog must be a sub-library. I have zillion folders and subfolders instead.
I've imported, I've processed, and I'm exporting processed files.
The export with "add" checked, and the misnamed (to me) "show subfolders" checked (LR confuses files and folders there. I want to see subfolders. I don't want to see the files in the subfolders displayed in the parent folder!), produces two folders and three of everything. Can't see the use of that! At least now I know why I kept getting two of everything in the original folder. As i spend my time using the files outside of LR there are no badges, just confusing multiplicity of pics.
Evidently I checked that "add" box long ago, probably per Kelby or something I was reading. It got unchecked recently, somehow.
When I export with "add to cat." unchecked I get what I showed elsewhere, with my "test." One folder with edited content and badges. Here it is again. That crescent moon was mostly sky before the edit.That's the edited version in the export. With badges.
If we're calling that the original, I'll have to "go ask Alice."
I cannot seem to explain any better the process, but I can see the problems you are having and the possible errors causing these problems, including-
Multiple catalogs,
All your problem identifications are correct, but: About the multiple catalogs - Not my doing! Not consciously. I leave that catalog stuff alone! (The dang things are mostly empty now, anyway!)
Multiple versions of Lightroom
Adobe sold 'em. I recall no warnings about anything. I probably thought I "owned" 2015 and might not want to rent Classic forever. Certainly since everything is CC, all about the cloud now, two LR programs would share the same data. Now I'm told the two catalogs are so different they won't speak to one another. I really don't know about that, because I'm sure the newer program didn't start with an empty catalog, it took the existing catalog somehow.
Not adding or importing ALL images to one catalog,
I've never created a single catalog, much less a second one. Wanted no truck with that. It all happened automatically in LR..
'Copying' at Import (as discussed by dj_paige , etc.
Possible occasionally. Doesn't explain the recently missing files. I did dj's search and the files are on the computer but not in LR. LR can't find'em.
Time for a coffee
You have done a herculean job explaining. And (or but) if I understand you correctly, you are saying that in this no-add case the export of the real NEW file goes back in the file with the original, the parent file (what would show as the parent file if "add" were checked), and OUT of LR, such that what was IN the catalog before the export (the file imported from and developed) is now OUT of the catalog because we did not "ADD" it to the catalog.
That terminology is highly problematic. What was IN is not OUT because we did not ADD. Also, if a normal "export," "add" or not, includes anything other that the edited files, LR has further serious issues with terminology.
With the process of import, develop, and export, the only normal understanding anyone could have of that process is that it is their work that is exported. That means edited files. You already have the original. You worked, you exported . . ..
Alice?
Time for a martini. In six hours. After responding to a few more posts, and doing something else, anything else. Maybe shovelling out a table feeder (for deer, corn and protein) that's been flooded since last October.
Your BIG mistake, is exporting files that are not really necessary.
Many Lightroom users only ever export to derivative files when they want one to - Give to a friend, Send in an email, Give to a print house, etc, and then they DELETE those derivative exported files from the computer. They are not needed again (with a few exceptions) and just occupy space in the computer.
Why? Because you can ALWAYS export a new derivative file from the original at any time. Remember the Catalog stores ALL your editing allowing you to Export derivatives from from the 'Original' file using any stage of edit History (or 'Snapshots' or 'Virtual Copies').
since I don't "do" catalogs, and certainly don't want to create one or head down that road at all.
This is where your "Thinking" is totally the wrong idea about Lightroom. You DO "do catalogs"
Lightroom does not work without a catalog! It will not even start without a catalog.
Lightroom is "DATABASE" software that records all information about your photos, including all the editing.
AND it will only show you the DATA that you 'Import' into the catalog, and that includes files exported.
It is NOT a File browser (like File Explorer) that shows ALL your folders.
I suggest you start your learning 'curve' again and read cover-to-cover: the free eBook from Victoria Bampton for Lightroom-CLASSIC: https://www.lightroomqueen.com/blog/
Also worth buying is her "Adobe Lightroom Classic The Missing FAQ" https://www.lightroomqueen.com/shop/
If I were to be in your situation, I would have EVERY photo imported into Lightroom to be shown in the catalog.
You can achieve this by selecting the topmost {Pictures} folder and doing a 'Folder Synchronization'. This will import every photo in sub-folders. (A Hint in Folder Synchronization: Do not remove 'Missing' photos until investigated). Only then will Lightroom have a full "picture" of ALL your photos (and sub-folders) that exist in your {Pictures} parent folder.
AdChoices