Sidecar Files (JPEG) to TIFF files are no longer recognized as such

Report · Nov 21, 2023

When exporting TIFF files as JPEG in the same directory and synchronizing that folder, the JPEG files are no longer recognized as sidecar files to the TIFF files.

Version: 13.0.1

Last working: 12.5.1

System: Windows 10 22H2

Steps to reproduce and behaviour:

Create a folder and put some TIFF files in it (e.g. img1.tif, img2.tif, img3.tif).
Create a new catalog and import that folder.
Export the TIFF files as JPEG files in the same directory.
Run "Synchronize Folder..." from the context menu: The Synchronize Dialog will show "Import new photos (3)" instead of "Import new photos (0)" as expected.
The following Import Dialog will show no new photos to be imported as expected. Click "Done."
Back in the Grid View the filename is shown as img1.tif (expected: img1.tif+JPEG) and the Sidecar Files list in the Metadata Panel [EXIF and IPTC] is empty (expected: "Sidecar Files jpg").

Remarks

A similar (broken) behavour applies to PSD and PNG files. However, the Import Dialog shows the sidecar JPEG files as new/separate files to be imported. For those files, this behaviour exists since I consciously use it (~ Version 11).
When importing TIFF files with the associated JPEG files present during import, the JPEG files are correctly recognized as sidecar files and as such are not imported separately. Also, the JPEG files are listed as sidecar files in the Grid View/Metadata panel
Impact: When moving images to a different directory, the exported JPEG files are left behind in the original directory.

Report · Nov 22, 2023

Why would the be sidecar files? they are different files.

And JPEF are never sidecar to raster images like TIFF files, they can be to RAW files, but only if they were the embedded JPEG within the RAW data (upon creation in camera)

Report · Nov 23, 2023

@GoldingD

Sidecar files are – by definition – files with the same base name but
different extensions as the root file.

So, when you rename/move/delete the root file, you want to
rename/move/delete the logically associated sidecar file(s) as well –
independent of the file format of the root file.

Currently, LrC recognized only raw files as root files. So, when you
rename/move/delete your PSD, PNG, TIFF (, etc) root file, any associated
sidecar file (like .xmp, .xmp_original, .jpg, .bak, etc) is left behind,
screwing your file organization: You have to rename/move/delete the
associated sidecar files manually from outside LrC, risking to forget or
miss some files and making mistakes. This can be particularly painful,
when you batch rename several hundreds of your files: You have you
rename each and every sidecar file (whose root file is not a raw file)
manually, with the risk of typos etc.

Report · Nov 23, 2023

I can see this conception of the job of sidecar files, as a bit problematic. A camera Raw+JPG pairing, I think we can all agree, is two files with the same origin and of the same nature. It makes sense both are affected by the same operations done from within the Catalog. But in normal LrC usage a TIFF would be a derived file with a different creation history. It's not so clear that you'd always want that and say a JPG with the same base name, to receive the same operation.

Furthermore, an export and a working file from which an export is made, are inherently of different natures from each other. This method of exporting to the same folder as the original and then syncing the folder is unnecessary: you can export selecting the option to "import to this catalog". Personally I see no reason or benefit to doing that re-import in any case: like many others, I regard exports as disposable ephemera. Not least, practically, to reimport an exported file will impair your ability to later repeat the same export, to reflect updated editing.

Also an export may require to remain on disk even after the same-name imported file has been removed from a Catalog, whether or not that removal involves file deletion. Personally I distinguish exports under a different file naming - I prefix the capture date - thus allowing a camera JPG to be exported as JPG and tell the difference. and even, to coexist within the same folder. I do recommend having exports not in the same folder though: LrC imported files and derived exports may demand different management. Of course I accept attitudes do vary on all of this.

Also: multiple Catalogs may refer in common to the same body of files, and the consequences of file 'set' operations of the sort discussed, may only become apparent in these other Catalogs much later on, or else simply get missed altogether.

Effectively, this functional locking-together of multiple files of differing natures, seems to me to offer more downsides than upsides. That's coming from the perspective of someone who regards all subsequent moving around and renaming within the file sytem, as an avoidable inefficiency. LrC permits those who think along those same lines, to simply forego file operations altogether - aside from the necessary initial ingesting from camera card.

Report · Nov 23, 2023

Seems to me that we are talking two different things…

From my understanding, it seems to me that you talk about the standard
case of out-of-camera-RAW file + out-of-camera-JPEG + xmp-sidecar files,
only, as it is described everywhere.

But the concept of sidecar files is much broader than this particular
case (in fact, I don't even have any out-of-camera-JPEGs where I have RAW files from my camera…):

Without loss of generality, you have 1. a source file (or root file, as
Adobe calls it), which can be a Raw, DNG, TIFF, PSD, JPEG, PNG, … file,
2. the processing settings (which exist in the catalog and perhaps in an
XMP file or inside that source file) and 3. the final output file, which
is usually a JPEG or TIFF or PNG, which you give to your
clients/friends/family/etc. (Yes, I know, there are people who promote
not to generate final output files…)

But there might be even more files which belong together. Here are some
examples:

* The Raw file out of the camera + the .xmp file + the output .jpg
* The raw TIFF files from the scanner + the processed JPEG
* The 2+ GB multi-layer PSD file + the final JPEG/TIFF/PNG
* Files converted to DNG (JPEG, Raw) plus a copy of the original
(.jpg) file as .jpg_original. So, you have the .dng + .jpg_original
+ output .jpg file

All those files associated with the source file are called sidecar files
by LrC.

Now, everybody has its own system to make himself unhappy. Mine is to
keep the final output file and possibly any other related files together
with the source (or root) file. I case I need one of them, I know where
to find them (the LrC Library module does a great job in finding them!):
They are all in the very same place as the source file is registered in
the catalog.

Once the different sidecar files to the source file are registered in
the catalog, LrC does a great job when moving/renaming/deleting the set
of files. Clearly, when you run, for instance, a batch rename on
hundreds of files, all the associated files are properly renamed as
well. When you merge or split directories, all the associated files are
properly moved to the destination folder as well.

The problem is just that LrC does not register the associated files as
sidecar files if the source file extension is .psd, .png – and now new .tif.

And probably another misunderstanding: I'm not talking about
re-importing the output files: When I mention "run Synchronize folder",
I mean that you have to run that procedure (uncheck "Import new photos")
in order to update the sidecar extension list.

The problem with LrC is that it contains lots of nifty but undocumented
features, which you find just by chance… This straight (but half broken)
handling of sidecar files is one of them…

Report · Nov 23, 2023

Thanks for the clarification about the purpose of Sync folder, that's interesting and makes some sense given your aims. So far as renaming or moving related files together, I do understand your logic to that extent.

But implications follow: I just see these as very limiting for Lightroom Classic working in particular.

Say you have an imported Raw for which different virtual copy treatments exist. Then you make an external edit from one of those. The Catalog can't simultaneously show multiple instances of the photo AND have them be in a sidecar-file relationship for file operations, at the same time. It has to be one way or the other.

Furthermore, if the identical base filename must be used in order for the sidecar relationship to work, you can have only one instance of each filetype in existence at a given time. The OS won't allow two identically named files in the same folder., and a file in a different folder can't be a sidecar.

It sounds to me as if there's an underlying wish, and yes this has nothing to do with sidecar files, to express the grouping of a set of files such that they will move to another folder / rename themselves intelligently while keeping their relation to each other. One such grouping could be a stack. And then a control option whether to "include similar imported files" - or some such idea. And a second control option "include similar not-imported files". What is to constitute "similar" may need definition, but I'd think that could probably allow for derived files having suffixes or prefixes, as well as for a variety of file types being involved.

Report · Nov 23, 2023

> It sounds to me as if there's an underlying wish

There is no underlying wish.

Adobe implemented this feature intentionally for good reasons (cf. above).

If worked PERFECTLY for Raw and .tif files. (Okay, virtual copies
didn't work for obvious reasons)

It was broken (at least since V11) for .psd and .png files.

Now it is broken for .tif files as well.

With this function broken, you have to move and delete the output files
manually. Renaming is only possible by deleting the output files and
re-export the files again, because you don't know their before/after names..

Report · Nov 23, 2023

Why are you trying to use jpegs as sidecars to tiff and psd, png files? Those file types store sidecar info in their metadata sections. I wish Lightroom supported actual sidecars to those filetypes but it uses the internal metadata sections in the actual metadata of the files and has done so since lightroom's inception. It only uses sidecar files (whether xmp or jpeg) for proprietary raw files. For dng, it also stores the sidecar info in the file itself. I know there was a bug some time ago where for tiff it recognized jpegs with the same name as sidecars but this happened because there are a few cameras that use the .TIF extension as their raw format and they fixed that bug finally (some people complained their jpegs disappeared) and now it only does that if the tif is actually a raw file from those cameras.