Copy link to clipboard
Copied
Env. is PSE 2020 on Windows.
See subject. I have a catalog--not my primary catalog--with 215 or so images. The CAXMPFiles folder for this catalog has 7,369 XMP files. Each of these relatively tiny files has an overhead of wasted file space in addition to cost to backup, etc.
Is this for real?!?! If not, why is Elements Organizer running amok here and how do I fix it?
I have another similar catalog with 480 or so images and 402 XMP files in the CAXMPFiles folder. My primary catalog has 33k images or so and 38,285 XMP files in the CAXMPFiles folder.
Edit to add: 7,200 or so of these were created in a several hour span yeterday morning. All I'd done was open the catalog, look around to see if it had an image I was remembering, and then close it.
Copy link to clipboard
Copied
Hi
Sorry about the incovenience you are facing. Sidecar file is created when you do any edit on the file and as you mentioned above, you are not doing any edits and still number of xmp files are getting created.
Could you please provide below details to know more on this problem and help you fix this.
You can share the files in private message if not comfortable here.
Regards,
Nidhi
Copy link to clipboard
Copied
The CAXMPFiles folder now has over 17,800 .XMP files. No end in sight. This is crazy!
The images in the catalog are, with only one or two exceptions, scans from a Nikon DS5000 or an Epson Perfection 636 a Canon LiDE80. Some are TIF, some are JPEG, some are DNG, some are SilverFast HDR TIFF, some are NikonScan TIF. (I have examples of all of these formats in other catalogs; those catalogs don't have this runaway XMP file creation.) None of these files has been edited anytime recently but this runaway XMP file creation seems to be a recent phenomena. I have tried catalog repair and optimize. Still thousands of XMP files, still thousands being created. There is only one recognized face in the people stacks. No image has more than two faces recognized; not all have even one.
I will provide several images and XMP files via PM ASAP.
Copy link to clipboard
Copied
Another data point: this morning, 2/5/2020, 10,410 of these files were created/modified in 2:23. That's more than one created per second. The last time any of the catalog image files was modified was 2/22/2019.
Copy link to clipboard
Copied
Anybody have any other thoughts? This catalog now has 17,801 .xmp files. No photos have been added or edited in this catalog in almost a year. But 17,681 of these were created this year.
What's the worst thing that happens if I just delete these things? (Besides more of them just get created.) What functionality or metadata do I lose?
Copy link to clipboard
Copied
I have a similar problem that also has nothing to do with the XMP files created by Adobe Camera Raw, whenever you adjust an image file.
70,000 XMP files in the CAXMPFiles directory that seem to have no connection to which Catalog I am using (I have several with 40-60,000 jpg, CR2 and videos files). 99.9% are all time stamped 12/29/20 when I upgraded from PSE 2018(ver16) to PSE 2021(ver19) and converted my catalogs.
Interestingly, CAXMPFiles directory for that same catalog prior to upgrading to PSE 2021 contained "only" 23,000 XMP files, so I gained an additional 47,000 files in the upgrade of the catalog. Side note, the previous XMP files were numbered 290212.xmp - 1581759.xmp, now 185.xmp - 2029236.xmp.
So I have deleted the first dozen .XMP files in the CAXMPFiles directory, to see if there was any impact. After several days of adding new images, editing new and existing images there is no apparent change in the quantitiy or timestamps in CAXMPFiles.
Also interestingly, two catalogs that contain pictures prior to 2014 and have also been upgraded to PSE 2021 have no CAXMPFiles directories at all.
Would love to delete the entire CAXMPFiles directory, but do not want to do it blindly - any thoughts?
Al
Copy link to clipboard
Copied
I wish I had a clue. Who knows what these files are necessary for or why they are necessary in tens of thousands for far fewer images than that. I have considered just deleting them and seeing if I can find any lost functionality. I've never done it.
Copy link to clipboard
Copied
I can't understand why nobody has given the answer about the CAXMPFiles folder.
It's used and necessary for the face recognition of people.
I don't use it myself; so I don't have those xmp files. Face recognition needs to record in some way where each face is located in each image. An image can hold many faces, so it's not unusual to have more xmp files than photo files.
However, the details given above suggest that old xmp data is kept or converted with each PSE version, probably without any cleaning or updating.
I'll leave face recognition users comment on the details, but I can suggest that it is easy to try what happens if you delete the xmp files.
First, you locate your catalog folder (menu Help >> system info of the organizer)
- you copy it with a new name elsewhere
- you test deleting the xmp
- you open the organizer by a double click on the catalog.pse19db file for your PSE2021.
Now, you'll see what you have lost in your face recognition.
I suppose you'll want to reopen your original catalog and delete the test catalog afterwards.
Copy link to clipboard
Copied
I saw this entirely on PSE2020. As noted, the images in question almost exclusively had one or no faces. (It's a catalog of images of one person.) Maybe 5% of them had two. Tops. 215 images. 17,801 XMP files created. Enough for 82 faces per image. Over 10,000 were created in less than three hours of file .XMP timestamps.
Maybe I should just dump them all and re-recognize the faces. Of course, it might run off into Adobe-la-la-land again...
Copy link to clipboard
Copied
Just to add to what Michel has said, I believe the CAXMPfiles folder contains the information for media analysis such as Smart Tags. It is my understanding that it is the json folder and .json files that contain the information for face recognition. So, I suspect that if you delete the CAXMPfiles, the smart tags will be deleted.
And I believe that Elements does a poor job of cleaning up the xmp files for images that have been deleted or where metadata has been changed.
Copy link to clipboard
Copied
I cross posted with you. I also want to add that my own catalog has approximately the same number of xmp files as visibile files in the grid (almost 200k).
Copy link to clipboard
Copied
There are a total of three tags in the catalog. No smart tags found. Images in catalog all have been in same catalog since pre- PSE 2020. Perhaps a few have had metadata changes after catalog was converted to PSE 2020. Ooops. Scratch that. Latest touch of any of these images was 2/22/2019 7:32 am.
Copy link to clipboard
Copied
Let me also add: PSE Organizer People shows 53 recognized images of one named person for this catalog. It shows 9 different clusters of a total of 10 different recognized "faces" of which 8 in 7 clusters are the same person, me. 2 in 2 clusters are not faces at all but are the same not face thing. And that takes >17,000 of the .XMP files to recognize that.
Copy link to clipboard
Copied
I created a new catalog from scratch.
In Media Analysis, Face recognition results in \FaceAnalysis\Json files and Analyze for Smart Tags results in CAXMPFile .xmp entries. Pictures with more faces seem to create larger .Json files.
My thanks to you all, I am happily off to purge 70,000 xmp files...
Copy link to clipboard
Copied
What's the result of your deleting xmp files? Did you delete the CAXMPFile subfolder in the catalog or the xmp files in that catalog?
In my tests on a catalog copy, deleting the files in the CAXMPFile resulted in in an organizer crash. Ulterior tests showed that deleting the CAXMPFile subfoler, plus Autocreations.1.db and AutoCreationCrashMonitorConfig.xml files did the job to clean up the catalog.
As mentioned above, to those tests on a catalog folder copy, not the real one!
Copy link to clipboard
Copied
No ill effects, have been flipping in and out of different catalogs, importing, exporting, editing photos - all good. I deleted all the XMPs inside.
Hours later last night, did notice that my CPU was runninng hard. Task Manager showed Auto Creations was consuming 8.6 gig of RAM, 30% CPU and there was a lot of hard drive churn. Lasted about 10 minutes, expected thousands of XMPs but no sign. Checked my Preferences, thought I had Media Analysis turned on by accident, but No, all turned off. Rebooted, no further sign of Auto Creations.
Copy link to clipboard
Copied
Very interesting and reassuring. A wild guess: maybe the strong activity was due to the updating of the Autocreations.1.db sqlite database to reflect the suppression of the xml individual files? I have no catalog to test this.
Copy link to clipboard
Copied
Just an additional note about the CAXMPFiles subfolder in a catalog.
You may have noted that in PSE2021, you have the ability to save the "structure" of a catalog, without the media files themselves.
You find the 'backup' under ProgramData/Adobe/Elements Organizer/Catalogs/_Autobackup_
Interestingly, the backup folder does not contain the thumbnail cache, which can be recreated at will. It does not either contain the CAXMPFiles folder. I conclude that in case of restoring the catalog, those xmp files are not restored. which may be a way to clean up those xmp.
I have found no tech or help doc about restoring the catalog but it does only require to rename the above folder as a new catalog; maybe some automatic process may trigger that if the catalog is found unusable.
Since the location for the Autobackup folder is common for all catalogs, I am not sure what happens if you backup different catalogs.
Is only the latest backed up catalog available or is there a way to know which is which?
Copy link to clipboard
Copied
Two data points:
1) I have done auto backups for three different catalogs. All appear to be stored in the ProgramData/Adobe/Elements Organizer/Catalogs/_Autobackup_ folder, each in a different subfolder with some kind of UUID name. If I had to tell which was which, I could do it by number of items in each of the subfolders. But that might not be as easy for, say, similar sized catalogs. MAybe peeking around in each could eventually tell you somehow? Agreed, use of these is pretty poorly documented, it seems.
2) Already having backups seven different ways to Sunday, and having not used that catalog most recently so I knew that PSE wouldn't be mucking in it in the background, I dove in and renamed the problem CAXMPFiles folder to CAXMPFiles WAS. Then I created a new, empty, CAXMPFiles folder, and opened the catalog. No crashes. All appeared the same. Even the handful of Smart Tags and their associated images. (I finally figured out where they are hidden...) Within a few minutes, I had 40 new .XMP files in CAXMPFiles. Half an hour later I still had 40 new .XMP files in CAXMPFiles.
I'm thinking whatever caused PSE 2020 to go berserk creating tens of thousands of .XMP files for this catalog will just have to forever remain yet another Photoshop Elements Organizer Mystery™.
Copy link to clipboard
Copied
Not much help on the question of the Autobackup, I do not use it. I rely on Macrium Reflect for daily backups of all my critical drives. But just "5 ways to Sunday".
My general comment would be that I wish Adobe was clearer about where it stored its temporary or "can be recreated easily" files, and give more control to the user to stick them on a spare "don't bother backing up" drive. Wishful thinking.
Copy link to clipboard
Copied
Not much help on the question of the Autobackup, I do not use it. I rely on Macrium Reflect for daily backups of all my critical drives. But just "5 ways to Sunday".
I also use another similar backup tool (Acronis) but that's not enough, and that does not take care of two situations:
- You want to move to another computer. The backup and restore process alone will ensure that restoring on a different computer, drive, partition or master folder will keep the links in the catalog to new location of the files.
- For some reason, only the catalog is not accessible or corrupt: the automatic autobackup each time you quit the organizer takes a backup after checking for errors. Most PSE users don't do a complete daily backup and they would not even know how to restore the catalog folder only with Macrium/Acronis.
My general comment would be that I wish Adobe was clearer about where it stored its temporary or "can be recreated easily" files, and give more control to the user to stick them on a spare "don't bother backing up" drive. Wishful thinking.
I understand what you mean and in the present case, I don't find any doc or help to explain where the catalog _autobackup_ file is stored and how it can be used. Udpating the online and pdf help is a bit late and I don't expect Adobe Staff to give answers in this forum. I mentioned this feature because it is related to the OP's question: how to get rid of xmp files. It could be a safe and easy way for non 'techy' users to do it without fiddling in catalog folders or backup folders.
As for your wishes, it's interesting to compare the choices in the Organizer and in Lightroom Classic. The organizer uses a backup workflow devised twenty years ago to ensure that the catalog (the folder) is restored with correct links whatever the new destination. With any other backup method, the restored catalog folder has to be 'reconnected' to the new destination. The reconnecting feature is inefficient for big libraries. The reconnecting feature of Lightroom is correct if you follow the recommendations. So, what I would expect (another wishful thinking) is that the organizer provides an efficient way to reconnect a big library. Why not use AI for the purpose, since Adobe boasts about such new features?
Copy link to clipboard
Copied
Since my system is result of 40 years of evolution, with many migrations to faster hardware, it is a complex arrangement of 10 partitions across 18 terrabytes. I always want to know where everything is and what its role is and can I safely purge it. Your description of Adobe's backup capabilities is interesting, I might have to play with it to see if it would be useful for the day when I decide to really
"clean house".