Exif Data of Virtual Copies

[#1]

Finding "duplicates", I am using the Teekesselchen plug-in and the well known ExifTool. Most of the duplicates are virtual copies, generated by the catalogue import process of Lightroom Classic. Most of them are identical, but some are quit different, for example different crop widths and crop heights. I really don't know where the Teekesselchen plug-in with ExifTool searches for Exif Data (master photo within the database? stored master photo itself? virtual copy within the database?), but I would like to know where LR Classic stores these Exif Data of virtual copies and how the tags are named.

Any hints would be very appreciated!

[#2]

Finding "duplicates", I am using the Teekesselchen plug-in and the well known ExifTool. Most of the duplicates are virtual copies, generated by the catalogue import process of Lightroom Classic. Most of them are identical, but some are quit different, for example different crop widths and crop heights. I really don't know where the Teekesselchen plug-in with ExifTool searches for Exif Data (master photo within the database? stored master photo itself? virtual copy within the database?), but I would like to know where LR Classic stores these Exif Data of virtual copies and how the tags are named.

Any hints would be very appreciated!

[#3]

Hi Maracujaniro,

Since this is the result of using this plug-in, wouldn't it be easier to ask them? I'm not trying to pass the buck here but I've never heard of this plugin but by definition, it's doing it's own thing "in" LR. I'm sure they would know what that thing is and how to work with it. At least I'd hope so!

[#4]

Thanks a lot, gary_sc, yes, I posted my question there as well.

[#5]

Just a thought, ”Virtual copies” are not image files or copies of image files, it's just data in the Catalog file.

[#6]

Thanks for your thought, DdeGannes, it's exactly how you mention, virtual copies are "just data in the catalogue file". For that reason I am interested "to know where LR Classic stores these Exif Data of virtual copies and how the tags are named".

[#7]

All data for a Virtual Copy is stored in the cataslog file only. No second XMP file is created and no entry in the XMP file for the original images is made.

The EXIF data for a Virtual Copy is Exactly the same as for the original. IT IS VIRTUAL. It does Not really Exist. It is all make beleive, Smoke amd Mirrors. Like Virtual Reality. It does not exist.

Virtual copies go hand in hand with the original. Delete the original and the Virtual copy goes Poof, Gone, Never to be seen again.

[#8]

Hi Just Shoot Me, you are right with "All data for a Virtual Copy is stored in the catalog file only". And exactly that's the problem: How can I make a duplicate finder plug-in (like Teekesselchen) reading and comparing these data instead of the Exif data stored in the image file itself? Any hint?

[#9]

What Exactly are you trying to find?

Duplicate images or images that Virtual Copies have been created for?

2 Completely different things. 

A virtual copy is NOT a Duplicate image.

Are you planing on writing a new plugin that would find images that have had VC created for? That already exists in the LrC Filter bar. It sure sound like that is what you are thinking of doing.

The Attribute section of the Filter bar has a icon with turned up corner, like the turned up corner of all VC's, to show you only Virtual Copies

[#10]

Thanks for responding, Just Shoot Me!

You are asking: "What Exactly are you trying to find?"

Well, I'll explain it more precisely.

Yes, I know, LrC has the option to only show virtual copies.

Here you see a small part of the result of the Teekesselchen plug-in, interpreting these images as "duplicates" of each other. Because I started the process only within virtual copies, no master image is shown.

What I am searching for is an ability to use more parameters or tags to distinguish between virtual copies, with different cropping sizes for example. So I'll have to know (1) where these data are stored (and unter which tag names) and (2) if Teekesselchen with ExifTool activated is searching in the LrC catalogue (and not in the Exif data of the master image). Maybe you have any idea?

Regards.

[#11]

I think the conclusion is simple. This plugin is not designed to do what you want it to do.

[#12]

Maybe, JohanElzenga, maybe. On the other side, if it were so simple, I probably would not have started this discussion thread, asking other people like you for help. 🙂

Both available plug-ins, Teekesselchen and Lightroom Duplicate Finder, offer the option to work with ExifTool. And ExifTool is very well known to be able to read every Exif tag. So my "investigation" didn't seem to be absolutely hopeless from the beginning.

Jeffrey Friedl (see post below) is giving hope.

[#13]

Lightroom edits are saved as metadata, but they are not EXIF tags. Jeffrey may be willing to write something, or adapt one of his plugins, that's another matter.

[#14]

Ok, JohanElzenga, so let me be more precise: I am searching for a plug-in able to recognize "duplicates" by EXIF tags or, in the case of virtual copies, by tags of LrC's database corresponding to EXIF tags. Correct?

[#15]

No, not correct. Like I said, Lightroom edits are not exif tags and they also do not correspond to exif tags. They are other kind of metadata. Exif tags is not another word for metadata. But let's stop this discussion, because it is fruitless. I hope Jeffrey can help you.

[#16]

"But let's stop this discussion, because it is fruitless." Yes. Regards.

[#17]

No doubt the plugin searches for capture date and time, among perhaps other things. The capture date and time of a virtual copy is obviously identical to the capture date and time of the original it belongs to, because a virtual copy is nothing more than a second set of metadata belonging to the same original.

[#18]

Thanks for your answer, JohanElzenga, you are right, "the plugin searches for capture date and time, among perhaps other things". These "other things" could be the crop width and the crop height of the virtual copy, for example. So, if I knew where these data are stored and which tag names are used, I possibly could use these data to avoid false duplicates.

[#19]

I doubt it will consider crop (and it would be wrong if it did), because crop and all other Lightroom edits are non-destructive edits. That means that no matter how much you cropped, the underlying image is still the original image. The goal of the plugin is not to find images that look the same, but to find duplicates: i.e. images which are the same. And a non-cropped image and a Lightroom-cropped copy of that image are duplicates.

The only debatable thing is that the plugin finds virtual copies at all. I think it should only find real physical duplicates. But perhaps finding virtual copies is an option?

[#20]

Thanks a lot, JohanElzenga, for your detailed explanation. Yes, I know that LR is non-destructive. And yes, I know "that no matter how much you cropped, the underlying image is still the original image". But we differ in the interpretation of "duplicates". For me, the original (non altered) image and a cropped virtual copy are not duplicates. And independent of a somehow acknowledged definition of "duplicates" (if there is any), I am searching for a method to avoid the original (uncropped) and virtual (cropped) image beeing treated as duplicates.

The cropped data (of a virtual copy) are stored in LR, and I would like to know (1) where these data are stored, I would like to know (2) how the tags (i.e. data fields) are named, and I would like to know (3) if any plug-in (like Teekesselchen) that uses ExifTool reads exactly these tags (instead of the tags of the original image). Not more, not less.

[#21]

It does not matter how you (or I) define duplicates. What matters is how the plugin does. The function of the plugin is to search for duplicate images. In that case it makes sense that it ignores non-destructive edits, including crop.

The  metadata are stored in the Lightroom catalog. If you know SQL, you could use an SQL editor to look into it. If you don't know SQL, I would strongly advice against that, because you could easily corrupt your catalog.

[#22]

Thx, JohanElzenga. The Teekesselchen plug-in has the option to ignore virtual copies or to not ignore virtual copies:

So to not ignore non-destructive edits makes sense as well, isn't it? But any further discussion of that "interpretation" will not help along.

Thanks for your hints regarding the SQLite database, that also mentioned ManiacJoe before (see below).

Regards.

[#23]

 I am searching for a method to avoid the original (uncropped) and virtual (cropped) image beeing treated as duplicates.

Export all of your virtual copies to a PSD, TIF or Jpeg and rename it. 

[#24]

Thanks, Randy_Hufford, yes, this could be an option. Another option is to manually exclude the corresponding virtual copies from the "duplicates", by any do_not_count tag, for example. The problem is to recognize these corresponding virtual copies within a large amount of images. It will be a hard job, especially because some of the virtual copies are bundled in a collection (together with its master image), others are not. So I hope Jeffrey_Friedl (see below) will find a better solution.

Regards

[#25]

Maybe of interest in this discussion:

An option in Teekesselchen Duplicate Finder-  

Duplicate Photos (Lightroom Queen)

[#26]

Thanks, WobertC! In Teekesselchen, I unchecked "Ignore virtual copies" because I want to get rid of all of the virtual copies created by LrC (when I imported images from other catalogues). Unfortunately, there are also virtual copies I created by myself, with different cropping sizes for example. The Teekesselchen plug-in compares only a few Exif data, most of them are identical in the master image and the virtual copies. As a result, nearly all of the virtual copies are recogniced as duplicates of the master image, these with different cropping sizes too. Therefore I tried to use ExifTool (see checkbox), but none of the tags I used worked well.

Your Lightroom Queen link I still read before, yes, a very useful and recommendable page, but without a solution of my problem.

Regards.

[#27]

I see your problem! And the Filter Bar-Attributes has the icon that shows only VCs, but it shows them all. It does not differentiate between 'yours' and the 'Catalog Import' VCs.

Following with interest.

[#28]

The LrC catalog is a SQLite database.

If you want, you can go to the SQLite website and download their database tools, which will allow you to browse the catalog as a SQL database, if you are comfortable doing so.

Adobe does not publicly document the organization of the catalog/database.

[#29]

Thanks, ManiacJoe! It's useful to know that the LrC catalogue consists of a SQLite database. If I knew that the Teekesselchen plug-in, with ExifTool activated, really compares tags of that database (and not only reads Exif data from the master image), I would try to read the SQLite database directly and hopefully find the tags I need.

But I'll see if Jeffrey Friedl can help me. His Metadata Viewer only reads data from the master image (and doesn't work with Teekesselchen, moreover), but Jeffrey should know the LrC catalogue very well.

Regards.

[#30]

It's a trivial matter for a plugin to compare two virtual copies and see that they are identical except for the copy name.  It's a bit more work to make sure that they are in exactly the same set of collections. If both are true, they can be considered pure duplicates.

If the plugin finds that a virtual copy is a pure duplicate of its master, or of another virtual copy, what would you hope to see happen? Simply have the spurious virtual copy removed?

[#31]

Wonderful, Jeffrey_Friedl, you are giving hope!

Yes, "if the plugin finds that a virtual copy is a pure duplicate of its master, or of another virtual copy", then I would like "simply have the spurious virtual copy removed", that's all.

Regards

[#32 · Accepted solution]

FYI, I've released an update to my Bag-o-Goodies plugin that includes the ability to find superfluous virtual copies.  It also lets you compare the editable differences between any two photos, which is perhaps the more-generally-useful feature.

Thanks to the original poser (maracujaneiro) for helping test initial versions of the plugin.

[#33]

That's great Jeffrey! I really love that plugin.

[#34]

I really love the two new options of your plug-in, Jeffrey_Friedl, as I love many others of your plug-ins, too! Thank you very much, you are simply the best!

Regards