How do I delete the jpeg from raw+jpeg imported images?

[$$PhotoHobby]

Hi,

Back in the day, I shot lots of images using raw+jpeg and imported them into Lightroom.  Now I realize those jpegs add no value and I'd like to recover the disk space that they're collectively occupying.

Is there a good way to:

1. Identify all images in a catalog which have raw+jpeg?

2. Remove just the jpeg for these images?

Thanks!

[Victoria Bampton LR Queen]

Yes, there is a way - sort of - but back up your catalog and photos first, just in case you make a mistake.  You may even want to work on a duplicate of your catalog, which you can delete when you've finished.

Go to preferences and confirm that the 'treat them separately' checkbox is unchecked so that they're imported as sidecar files, as they are at the moment.

Run an import on all of the folders, just to double check you've got all unique photos imported already.

Put all of the current photos in a collection so you can easily identify them if you make a mistake.

Go to preferences and check the 'treat them separately' checkbox so that it'll import the JPEGs separately in the next step.

Now go back and import the same folders, and it'll pull in all of the JPEGs as separate photos.

They should all be in Previous Import.  What you do at this point depends on how certain you are that the newly imported photos are just the raw+jpeg JPEGs.  You might select all and delete them, or you might choose to mark them with a colour label so you can skim through All Photographs and double check before deleting.

Obviously it's not quite as clean cut a solution as you might like, but it works.

[$$PhotoHobby]

Hi Victoria,

First, thanks very much for taking the time to answer my question. The approach you're suggesting should (hopefully) work but it's one heck of a hack so I'm a bit reluctant to do it.  I need to be 100% confident that I won't be accidentally erasing any jpegs for which there are no corresponding raw files (I have some jpegs without raws from cameras which can't shoot raw).  The thing that gets me is that LR already allows you to search for images based on file type, (but not raw+jpeg today).  It seems that it would be a useful feature to be able to search for raw+jpeg and then filter out raw (or jpeg) from the selection so that you could erase / process / do whatever you need with the selection.  Maybe something for LR5?

Sean

[Cornelia-I]

Filter the previous import collection of jpgs for camera, and select NOT the cameras which do not save raw format.

Then you can be sure that you do not accidentally delete one of those w/o a corresponding raw.

Cornelia

[Victoria Bampton LR Queen]

$$PhotoHobby wrote:

Maybe something for LR5?

Yeah, it's a request I've heard enough times, so it would get my backing.  Put in a request at the Official Feature Request/Bug Report Forum

[Todd Shaner]

This might help you:

http://www.qandasystem.info/photo/how-to-delete-jpg-files-but-only-if-the-matching-raw-file-exists/

As long as you haven't changed the file raw/JPEG paired file names a script should work. If you have changed naming it just won't delete those JPEG files.

[martin-s]

Hi Victoria,

thanks for your precise instructions. I never had to deal with this issue myself, but have been asked a few times.

A few questions…

I suspect a simple smart collection to search for file type is JPEG wouldn't work in this case, since the images had been imported as RAW+JPEG, right?

What would happen if you deleted the associated JPEGs behind LR's back in the Finder? Would it throw up a warning about missing images even if the RAW file was still there?

Martin

[Victoria Bampton LR Queen]

If you followed the instructions for importing the extra files separately, the file type as JPEG would work fine as long as you can be absolutely certain there are no other JPEGs that you intended to keep.  I know I end up with JPEGs from my iPhone in my catalog, which I wouldn't want to accidentally delete, hence the extra step of the collection and previous import for security.

Deleting the associated JPEG - I haven't tried it for a long time, so I can't remember offhand.  The issue would be finding them all, without accidentally including JPEGs that weren't part of a pair.

[Todd Shaner]

This LR plugin looks like it will let you find raw+JPEG pairs:

http://regex.info/blog/lightroom-goodies/extended-search

It claims to be LR4 compatible. I haven't used it, but try this search criteria:

Find images whose: Leave it blank or select something that will pull up all images (i.e.wildcard). I don't know what's available.

JPEG file: Select 'has related JPEG' and 'include related JPEG in search'

Once you have all of the files filtered you can review the results. You should be able to sort by 'File extension' to separate JPEGs from raw, and then move them to another folder on your hard drive or simply delete them.

Give it a try and let us know if this works for you. Looking at this now I have no idea why Adobe didn't include it as an option in the Filter Bar.

[areohbee]

@trshaner - I think jpegs would have to already be imported separately for that to work, no?

PS - running SQLiteroom with this as a script will make a collection of raws with jpeg sidecars:

select LibraryRootFolder.absolutePath || LibraryFolder.pathFromRoot || LibraryFile.idx_filename as fullFileName, LibraryFile.sidecarExtensions

from Adobe_images as Image,

AgLibraryFile as LibraryFile,

AgLibraryFolder as LibraryFolder,

AgLibraryRootFolder as LibraryRootFolder,

Adobe_AdditionalMetadata as AdditionalMetadata

where Image.rootFile = LibraryFile.id_local

and LibraryFile.Folder = LibraryFolder.id_local

and LibraryFolder.rootFolder = LibraryRootFolder.id_local

and AdditionalMetadata.image = Image.id_local

and Image.masterImage is null

and LibraryFile.sidecarExtensions in ( 'jpg', 'JPG')

order by fullFileName

;

That would take care of part 1 anyway, so you'd have the subject photos identified, and then you could deal with them using Victoria's procedure.

Another idea:

If somebody could come up with the SQL to divorce all raw+jpeg pairs, then this would work:

1.  Synchronize root folder, and re-import all photos, so nothing is missing.

2.  Run script to divorce raw+jpeg pairs (shutdown first, run on dormant catalog, then restart).

3.  Synchronize root folder, and re-import all photos

4.  Delete everything from disk that got imported.

Backup first of course...

*** NOTE: This procedure would NOT work for me, since it would include jpegs edited externally and given same base name as source, to avoid problems with published copies... But if you don't do that, it might work: do this kinda thing with caution and at your own risk.......

Rob

[Todd Shaner]

Rob Cole wrote:

@trshaner - I think jpegs would have to already be imported separately for that to work, no?

Rob you are correct, but the OP didn't specifically state that the JPEGs are not already imported (i.e. 'Treat JPEG files next to raw files as separate photos' checked). Even they aren't imported it looks like it might be easier and "cleaner" to import them, and use an "extended search plugin like Jeffrey Friedl's. Then you don't have the NEF+JPEG file extension label issue. If Adobe adds this capability in a future release orphaned NEF+JPEG extension tags could create a problem using it. Using 'Synchronize Folders' with 'Show Folders in Subfolders' selected the JPEG import should be fairly quick with 'Render Previews' set to 'Minimal.'

Rob & Richard keep the suggestions coming, since I don't expect Adobe will add this capability any time soon.

[areohbee]

As Richard has pointed out, it's mostly an aesthetic issue, but I personally don't like having wrong info staring me in the face (and embodied in the database) - best to fix it right... - I'll let ya know if any light bulbs turn on...

PS - I assumed they were imported with "Treat separately" off - e.g. OP didn't object to Victoria's comment:

|> "Go to preferences and confirm that the 'treat them separately' checkbox is unchecked so that they're imported as sidecar files, as they are at the moment."

~R.

[richardplondon]

Rob Cole wrote:

As Richard has pointed out, it's mostly an aesthetic issue, but I personally don't like having wrong info staring me in the face (and embodied in the database) - best to fix it right...

Sync Folder will fix this - one can un-select "check for metadata updates" and also forego importing any new images found, so as so simply refresh the Catalog's information wrt the files present.

RP

[areohbee]

So, I've discovered some weird things - Lightroom has some inconsistencies (bugs?) in file handling / catalog updating.

I've had both of these things happen:

* If I delete the jpeg sidecar from disk, the badge persists NEF+JPEG even after a folder sync.

* If I delete the jpeg sidecar from disk, the +JPEG is cleared from the badge after a folder sync, leaving just NEF.

Hmm... - I did notice that it was incrementing the suffix upon import sometimes too, even though the file being imported was NOT in the catalog. e.g. importing A.NEF as A-2.NEF, even though A.NEF was not in the catalog, nor on disk (nor A.JPG). I guess we all get confused sometimes - so does Lightroom .

Anyway, this will delete extraneous jpeg sidecars with a single click (and a confirmation prompt...):

RawPlusJpeg

~R.

[Geoff the kiwi]

Please gentlemen, if you want to get into a discussion on subjects not directly related to the original question please start another thread. This thread has useful information but I believe you've left the Op in your creative solutions cloud. Stick to the point is what I mean I guess!!

[areohbee]

Most everything seems relevant to me. This *is* a user-to-user discussion forum after all. It therefore seems fitting for users to, discuss, the subject...

Note: all of this discussion ultimately lead to a nearly perfect solution, which has been laid on a platter for the OP in my previous post.

Perhaps Adobe should consider opening a new forum just for "Customer Support", where only Adobe employees and "MVPs" can respond, or at least "LVPs" replies require moderation.

[areohbee]

Geoff the kiwi wrote:

...I believe you've left the Op in your creative solutions cloud...

So, where you be, OP?

[$$PhotoHobby]

My apologies for the hiatus but rest assured, I've found the thread to be quite educational.  Thank you all for taking the time to look into this.  I too have poked around a bit and discovered that LR is definitely storing information about the JPEG associated with the RAW/NEF/... in the LR database.  It does not appear to be stored in the sidecar .xmp metadata file.  So any "safe" changes would have to be done by some utility that understands the database schema and/or uses the appropriate LR APIs for selecting and removing files.  So an external script that finds and removes the JPEGs outside of LR might leave LR in a confused state.  Rob Cole's post #18 seems to indicate that LR doesn't handle the disappearance of the JPEG consistently (possibly a bug).

From all I can tell, LR doesn't provide any way in the UI to select and manipulate the JPEG associated with a RAW/NEF/whatever.  I've not looked at the APIs to see if there's a way to do this programmatically.  If, there isn't, I'd be *very* worried that LR might not do the right thing if the JPEG just disappears as the lack of API support suggests to me that this is a case that the LR team probably hasn't put much thought into or tested.

It seems that Victoria's original suggestion (painful as it is) is probably cleanest as it is, in effect, a safe re-import, but with JPEGs treated separately.  Apart from having to then manually match JPEGs with RAW/NEF/whatever and clobber the JPEGs, I'm guessing I'll lose all the metadata associated with the RAW image (correct me if I'm wrong).

So my "solution" (for now) is to make a request for the feature enhancement and hope that it shows up in the next release.  Unless I'm totally confident that the LR catalog database would be cleanly updated, I'm very hesistant to take the chance on any 3rd party script.

[areohbee]

$$PhotoHobby wrote:

I'm very hesistant to take the chance on any 3rd party script.

Just so ya know, what the deletion feature in RawPlusJpeg does is ultra simple:

1. Identifies extraneous jpeg files.

2. Presents them for you to review.

3. Deletes the extraneous jpeg files, if you approve. (puts them in the recycle bin if possible).

It does nothing to the database nor anything else.

You sync folders with 'Treat JPEG files next to raw files separately' afterward, which is what sets the database right.

The only potential for trouble is the criteria for determining which jpegs are extraneous.

It has no direct access to the database, so what it does is check for jpegs with same base name as raw (which *must* be present in catalog, and on disk), that aren't in the catalog, which includes the jpegs associated with RAW+JPEG (imported as a unit), *and* any jpegs that were imported separately, but subsequently removed from the catalog, or had never been imported.

So, what it does will be the perfect thing to do, *if* you first synchronize folders (with 'Treat JPEG files next to raw files separately' checked) to pull any jpegs in to the catalog that were previously imported separately, but may have been inadvertently removed from the catalog (or had never been imported), before invoking the delete feature of the plugin.

Note: You must do that for any manual procedure too, or you will have the same problem, except in the manual case, you won't have the benefit of the plugin checking your work, and presenting the files to be deleted, before you commit.

It's your call, obviously, and to save Geoff the Kiwi from pointing this out: the plugin comes with no guarantee, warranty, or any other kind of tea.

Consider backing up catalog and photo files before doing this maintenance, whether manual, or plugin-assisted.

In my opinion, the plugin-assisted method is much safer than the manual method, which is the only reason it was written, *not* to save steps, although it does that too. For example, you can not, with the plugin, delete any jpegs that do not have a corresponding RAW in the catalog and on disk, period. - unless there is a bug of course, but there probably isn't , at least not one that would change the aforementioned assertion... - it wouldn't take much of a bug to cause it to barf, but it would take a big bug to cause it to delete files that weren't on the list, etc...

Seriously, I don't think you have to wait for Adobe on this one. I wouldn't (I didn't - my catalog is all clean now!...).

PS - there is *very* little recorded in the database about the jpeg sidecar - it's existence upon import (when "as a unit") - that's about it. Most of the funny business about updating that we were fumbling with above, stems from the software (and the person at the keyboard), *not* the database. Syncing folders with 'Treat JPEG files next to raw files separately' checked, once the jpeg sidecars have been cleaned up, *will* set the database straight!

Cheers,

Rob.

[richardplondon]

Rob Cole wrote:

PS - there is *very* little recorded in the database about the jpeg sidecar - it's existence upon import (when "as a unit") - that's about it. Most of the funny business about updating that we were fumbling with above, stems from the software (and the person at the keyboard), *not* the database. Syncing folders with 'Treat JPEG files next to raw files separately' checked, once the jpeg sidecars have been cleaned up, *will* set the database straight!

If I may just emphasise a few points - we have been discussing two options: delete the unwanted JPGs from outside LR, and delete the unwanted JPGs from inside LR. Some utilities have been mentioned for programmatically finding just those JPGs which share a name with a Raw in the same folder. If one regrets that one shot Raw+JPG in the first place (or if, as with one camera I use, the camera offers no way to take Raw without a JPG also) - then one may want to remove all "paired" JPGs, including those paired with Raws that have not been imported to LR. So that may incline toward a file utility of some kind that is run outside LR.

The only slight adverse consequence of the external route will be, when LR incorrectly continues to report "Raw+JPG" against given images. Sometimes that seems to fix itself, which is far more puzzling than either always, or never (grin).

But it is worth pointing out that there is absolutely nothing wrong with the Catalog in this state - we still have the functional situation that we wanted, there is no internal database corruption and it is not failing to work properly. It is simply mistaken - out-of-date - in respect of some external information. Having actively decided to eliminate such JPGs, we are by definition not relying on them - we are never going to be in any material way disappointed because we thought there WAS a JPG there, and it turns out that there ISN'T.

LR itself doesn't even care sufficiently about these partner JPGs, to notice - let alone, complain - when they are missing! If we conclude that it should care more, then a very small amount of change should suffice - for example, if it self-updated reliably a lot of this later discussion here, would have been redundant.

The Sync Folder step described in recent posts, just to reiterate, does not need to actually do anything beyond an "inventory check" on the accuracy of the Catalog's records about the file system contents. One can just decline any offered Imports. Then all will be verified as correct.

Personally, I would like to see a third option presented for treating paired JPGs. We currently can treat them either "visible/editable", or "invisible/non-editable". Adding another option checkbox: "exclude from import/delete during Folder Sync", would be IMO sufficient. If the Delete choice brought up a standard Delete from Disk / Remove from Catalog dialog, everyone's wishes could perhaps be cleanly met, and inside the standard LR paradigm.

[areohbee]

richardplondon wrote:

we have been discussing two options: delete the unwanted JPGs from outside LR, and delete the unwanted JPGs from inside LR.

Note: Deleting the unwanted jpegs from inside Lr, via RawPlusJpeg, is equivalent to deleting them from outside Lr, since the jpegs to be deleted are not in the catalog. Put another way: if they are not in the catalog, whether a plugin deletes them or OS - same difference.

[richardplondon]

On a far more low-tech note: if some means are found to select and delete the JPG in a Raw+JPG pair outside LR, this does not cause LR itself any problem. The action does not NEED to have been carried out from inside Lightroom or a LR plugin, IOW.

Removing the JPG externally does not result in any "not found" status, since this file has never had any separate representation in the LR Catalog anyway.

In due course, LR will merely update its filetype display - from (e.g.) "PEF+JPG", to (e.g.) "PEF" - so as to reflect the current physical situation on disk. The same happens in reverse, btw, if you manually add in a same-named JPG, on disk, where there was not one before.

File moves or renames carried out from inside LR will involve any same-named JPG found, automatically, too.

[areohbee]

Hmm - I just double-checked: indeed, Lightroom 4 does not complain about missing jpeg sidecars, like Lr3(2?) did (if I remember correctly).

Still, I could not get it to forget about the sidecar (always says NEF+JPEG on the thumbnail on the grid), and always says "Sidecar Files: JPG" in the metadata panel. Is there a trick I'm missing? only happens that way with PEFs?? Are you on Mac??? (me: Windows).

Also, in my testing, jpeg only follows raw when moving/renaming, if jpg is considered to be a sidecar, not separate.

Rob

[richardplondon]

I don't know what triggers LR to update its filetype label, after externally deleting the +JPG., but it seemed to do so automatically after a short delay - I am on Windows. This was in a (LR4.1 final) Catalog where (btw) I do not have auto metadata writing enabled.

Also, in my testing, jpeg only follows raw when moving/renaming, if jpg is considered to be a sidecar, not separate.

Exactly so, that is what I was assuming too. If the JPG is separately present in the Catalog, then yes LR will compain about a missing file if this is removed - and there will be a separate thumbnail and other previews - and LR will store develop and metadata, collection, stacking etc settings independently for this image. Also it can be moved and renamed independently so far as LR is concerned.

I do consider the term "sidecar" for a +JPG file, potentially confusing - since this already has an established conventional usage in referring to XMP metadata-only files (which do not contain image data that could be separately imported to LR). It's true that the behaviour is very similar. Maybe we can call these silent +JPG versions "secondary" image files instead - as distinct from the "primary" status of either a Raw file, or of a JPG file which does appear independently.

[areohbee]

richardplondon wrote:

I don't know what triggers LR to update its filetype label, after externally deleting the +JPG., but it seemed to do so automatically after a short delay - I am on Windows.

Huh. Not behaving that way for me, Lr4.2RC1, e.g. as shown in the badge:

JPEG "sidecar" has been deleted from disk for a long time.

Are we talking about the same thing?

~R.