P: Lens Corrections defaults not correct if Previews are "Embedded & Sidecar"

[Nicholas_AJG]

I'm testing with a Canon RAW files from a EOS 7D with EF 70-200 L IS USM lens.

I have set the Develop Default Setting for this camera to enable "Remove Chromatic Aberration" and "Enable Lens Corrections".

1) When I import the files and have Build Previews set to "Minimal" the 2 check boxes ("Remove Chromatic Aberration" and "Enable Lens Corrections") are checked and the correct Make, Model and Profile are found.

2) When I import the files and have Build Previews set to "Embedded & Sidecar" the 2 check boxes are checked BUT Make, Model and Profile are "None". HOWEVER if I click on the photo, go to the Develop module, click on a different photo in the filmstrip then click on the original photo the correct Make, Model and Profile are displayed most of the time.

I also noticed if I click Auto tone or if I adjust a slider and then hit Command-Z then the correct Make, Model and Profile are displayed.

[JohanElzenga]

I assume that's because 'Embedded & Sidecar' means that the preview you are looking at is the embedded preview that the camera added to the raw file. Lightroom's lens corrections are obviously not applied to that preview. Only when Lightroom has created its own preview, the lens corrections are applied and visible.

[Todd Shaner]

I can confirm what you are seeing using LR 7.5 on Windows 7 with Canon 5D MKII and 6D cameras with 17-40mm F4L and 16-35mm F4L IS lenses. With 'Embedded & Sidecar' Import Preview setting the Lens Corrections panel appears as below and has the error message 'Unable to load lens profile.' Interestingly when I switch back to the Library module the image displays with the Lens Profile and other 'Default Develop Settings' applied and is no longer using the Embedded Preview. This is without applying ANY setting in the Develop module.

So it appears there are two issues. It's my understanding that when using Embedded & Sidecar Import Preview setting the Library Preview will not be updated until Develop module settings have been applied. Merely viewing the image in the Develop module without applying any adjustment settings should not cause LR to discard the Embeded & Sidecar Preview and rebuild a new preview.

The only way I am able to "repeatably" get the Lens Profile to be applied in the Develop module is to click on the Reset button (applies my default settings) or Sync the Lens Profile setting across images.

[Todd Shaner]

Johan, did you actually try this for yourself and Import raw files with the File Handling> Build Previews setting 'Embedded & Sidecar' selected? Please see my post below and tell us your results using this Import setting and your 'Default Develop Settings' with 'Enable Profile Corrections' checked.

[JohanElzenga]

Yes, I always import with 'Embedded & Sidecar' to speed up import. After the images have been imported I let Lightroom generate its own previews. Lens corrections and chromatic aberration are then applied automatically and correctly.

I have to admit that this could be because I also apply a develop preset on import.

[Todd Shaner]

"I have to admit that this could be because I also apply a develop preset on import."
That's not the same type of Import we are discussing here. If you apply a 'Develop Preset' in the Import module LR will automatically build standard previews. This also negates the speed benefit of using 'Embedded & Side car' previews and will prevent you from seeing the issue discussed here.
To duplicate this issue you will need to:
1) Change your 'Default Develop settings with 'Enable Profile Corrections' checked.

2) Import with NO Develop Setting selected in the Import module and 'Embedded & Sidecar' Preview selection.
Once the import has completed confirm you can see the double-arrow icon on the filmstrip in the upper left-hand corner of the thumbnail. Then switch to the Develop module without applying any settings and view the Lens profile panel. What do you see?

[JohanElzenga]

No, Lightroom will not automatically build standard previews. It will only do that when I tell it to. The images come in with Embedded previews, and this stays that way until I click on the double-arrow icon in the upper left corner to tell it to build standard-sized previews. Or they will be built when Lightroom is idle (if you set that in the preferences). Maybe I won't see the issue you are discussing, but please don't think you can tell me how my own workflow works.

[Nicholas_AJG]

Todd,

So you agree this looks like a bug? I'm new here, do I need to do anything so Adobe is make aware of this?

Thanks,
Nicholas

[Todd Shaner]

I never apply Develop presets on Import, but just tried it. Using 'Embedded & Sidecar' preview setting and a Develop preset that applies 'Lens Profile' and 'Chromatic Aberration' settings the Embedded preview is retained as you stated. My apology! However, I still see the issue being discussed her. When I switch to the Develop module 'Lens Profile' shows as 'None' and message 'Unable to load lens profile' (see screenshot below).

Please try the test as I outlined in my last reply and see if you can replicate the issue. Thank you!

[Todd Shaner]

Nicholas, I do agree it is a bug and this is the correct place to report it. I've added my 'Me To' which is the best way to get Adobe's attention.

[JohanElzenga]

I’m sure you see what you see. I don’t doubt that. As I’m typing this on my iPad, I cannot verify your findings. All I can say is that in my workflow I do not have any problems of lens corrections not being applied, probably because the preset forces Lightroom to update its rendering (it also applies auto tone). I never get into the situation where I have switched to the develop module and then still see the embedded preview. When I switch to the develop module, I see a Lightroom rendered preview (with correctly applied lens corrections).

[JohanElzenga]

“Merely viewing the image in the Develop module without applying any adjustment settings should not cause LR to discard the Embeded & Sidecar Preview and rebuild a new preview.”

Actually, it’s a bit more complicated. The develop module uses its own previews, which are not the same as the previews in the Library and other modules. So switching to the develop module should always cause Lightroom to show you a new preview. The Develop module should never show the embedded preview. However that new Develop preview does not replace the (embedded) one of the Library module either. The Library preview is only re-rendered when you have applied edits, and I think that simply switching to the develop module is already seen as such (perhaps to avoid confusion).

[Todd Shaner]

"The Develop module should never show the embedded preview."

It's not showing the embedded preview in the Develop module. Just to be clear it correctly shows the Develop preview, but without the Lens Profile applied.

"However that new Develop preview does not replace the (embedded) one of the Library module either."

When switching back to the Library module with NO settings applied in the Develop module the Library module creates a standard preview with the 'Default Develop Settings' applied and the 'Embedded Preview' is removed.

"The Library preview is only re-rendered when you have applied edits"

In previous versions of LR (prior to 7.5) the Embedded Preview would remain in the Library module until an actual Develop setting adjustment was applied to the image.

"and I think that simply switching to the develop module is already seen as such (perhaps to avoid confusion)."

This is actually not the issue of concern here and if Adobe changed this behavior it is just an undocumented change. I'll take it either way!

Just to be clear the real issue is that when using 'Embedded & Sidecar' the LR 'Default Develop Settings' for Lens Profile are not being respected when 'Enable Profile Corrections' is a part of those settings. In fact an error condition is created, "Unable to load lens profile."

[johnrellis]

After poking around at this for quite a bit with Todd, looking at Lens Profile field values via the plugin SDK, I think this bug is related to two other long-standing bugs:

- When you sync Lens Profile Corrections with a batch of photos, the SDK returns invalid values for the Lens Profile Name until you force the previews to be re-rendered by scrolling or rebuilding previews.  
https://feedback.photoshop.com/photoshop_family/topics/lightroom-sdk-photo-getdevelopsettings-return...

- When you apply Auto to a batch of photos, subsequent changes to Develop settings can be lost unless you first cause the previews to be re-rendered by scrolling or rebuilding previews.
https://feedback.photoshop.com/photoshop_family/topics/applying-auto-tone-to-a-batch-of-photos-loses...

It appears that LR has an internal architecture where for some batch operations like Auto and assigning lens profiles, it marks the photos with an annotation that the operation should be computed "when needed" in the future, but doesn't actually do the computation. This allows the batch operation to complete quickly. But other parts of LR forget to check for that "when needed" annotation and get confused.

[Nicholas_AJG]

I have tested with a single Canon 5D4 RAW file and get the same results, even when Previews is set to Minimal.

Import RAW file. Switch to Develop module get "Unable to load lens profile." error. Switch to Library and navigate to a different folder. Go back to original file and switch to Develop, lens profile information is now filled in and the error is gone.

[Todd Shaner]

I checked my LR catalog using John R. Ellis Any Filter plugin and it appears running 'Build Standard-Sized Previews' "fixes" the Embedded & Sidecar and Minimal Import Preview  issue and the other two issues he mentions above.

In addition any image files that have outdated or no Standard preview should be rebuilt anyhow to maintain performance. Files that are up to date or have never been edited in the Develop module will not be rebuilt and don’t affect the preview building overhead. Here's how:

1) To speed the preview building process go to menu Edit> Preferences> Performance and check 'Generate Previews in Parallel.' Close and restart LR to apply this setting change.

2) In the Library module Catalog panel (left-side) select 'All Photographs.' This will allow checking and rebuilding previews for ALL of the files in the catalog.

3) Go to menu Library> Previews> 'Build Standard-Sized Previews.' This can take hours and will make your system sluggish so you may want to run it overnight.

NOTE: If LR appears sluggish when editing images after completing the preview building process you may want to uncheck 'Generate Previews in Parallel.' Try it both ways and decide for yourself.

Going forward the ‘Standard’ Build Previews selection should be used in the Import module until this issue is fixed.

[JohanElzenga]

I disagree with your last conclusion. Importing using ‘embedded & sidecar’ still has the advantage that the import process itself is much faster. After the images have been imported, you can click on the double arrow icon of one of them, and then select the option to build standard-sized previews for all these images. You may have to apply a develop preset to get it to work correctly, so here is the trick: create a develop preset that applies the lens corrections and use that. As I said at the beginning, this is how I work and I do not experience the problem of missing profiles.

[Todd Shaner]

I'm reluctant to suggest using 'Embedded & Sidecar' since the Lens Profile issue being discussed here appears related to the issues John R. Ellis  mentions above. I feel it is prudent advice. If you or anyone else has confidence that it's NOT causing any problems  on your LR system (including the ones John mentions) then by all means go ahead and use it.

I tested both Embedded & Sidecar Preview Import with Lens Profile excluded from the Default Develop Settings and then Standard Preview Import with Lens Profile included. Embedded and sidecar was actually significantly slower (34%) with 'Generate Previews in Parallel' NOT checked. With it checked Embedded and sidecar was slightly faster (5.3%), but not much! I'm sticking with 'Standard' Import Preview Building setting. YMMV, so test it for yourself.

71 Canon 5D MKII CR2 (21 Megapixel) raw files with 2560 px Medium Standard Preview

Generate Previews in Parallel NOT checked
Embedded & Sidecar w/o Lens Profile: 63 sec.
Standard Preview with Lens Profile: 47 sec.

Generate Previews in Parallel checked
Embedded & Sidecar w/o Lens Profile: 38 sec.
Standard Preview with Lens Profile: 40 sec.

[DMEEPhD]

I've seen this 'bug' for years and that's why I never use LRs lens profiles.  I open images I wish to post-process in DxO (from within LR as an external editor), apply the DxO profiles—which are far superior IMHO—to the raw files, and close DxO.  The resultant raw files goes back into LR with an attached .dop file and the lens corrections remain in place.

Sometimes one needs a specialist application like DxO to do what a generalist application like Lightroom can't hope to accomplish.  (I have also found that DxO updates their lens/camera combinations far more often than does Adobe.)

[JohanElzenga]

Interesting observation. It seems that something has changed in the import sequence recently, probably in LR 7.5 in fact. I always add the imported photos to a collection, and use the moment they appear in that collection as the moment of 'being imported'. Because stacks are local, I want to work with them from within that collection.

Until recently, Lightroom would first generate all the standard previews, and then as the last step they would appear in the collection. That made import very slow, and 'Embedded & Sidecar' made it much faster. Now the sequence seems to have changed. I see the images appear in the collection and then the previews are being updated. That makes the difference between Standard and Embedded indeed much smaller. I timed it for 30 Sony A7R III images and see the same difference of 'Embedded' now only about 10% faster.

Good to know, thanks. That makes Embedded & Sidecar a lot less useful (pretty much useless in fact).

[Todd Shaner]

"Until recently, Lightroom would first generate all the standard previews, and then as the last step they would appear in the collection. That made import very slow, and 'Embedded & Sidecar' made it much faster. Now the sequence seems to have changed."

I remember the same behavior when first using Embedded & Sidecar Preview workflow with LR Classic 7.0 back in October 2017. I didn't run any tests to compare it against Standard Import Previews because it was clearly much faster. It would still be useful if you have to review a shoot at 1:1 Zoom view before editing since building 1:1 Import Previews will take much longer. Begs the question as to what changed and is it a bug?

[JohanElzenga]

It seems to me that importing with standard previews is now much faster than it used to be, so if that is a bug, then I would like to see a lot more bugs like that! 🙂

[Todd Shaner]

"It seems to me that importing with standard previews is now much faster than it used to be"

Agreed. It used to take ~2.0 sec to build a Standard Preview with my Canon 5D MKII 21 Megapixel raw files. Using LR 7.5 it takes <0.7 sec. with the same 2560 px Medium Preview settings. If Adobe did something to speed-up Standard Preview building that slows down Embedded & Sidecar I'd call that a bug. I didn't run a benchmark previously using Embedded & Sidecar Previews so it's hard to say for sure something has changed, but It seems slower. If it's not much faster than using Standard Import Previews the only benefit is if you need to do extensive 1:1 Zoom review before editing. I don't think that was Adobe's intention so it looks like a bug may have been introduced.

[Rikk Flohr: Photography]

Thanks Todd, for the videos. Please leave them posted for the team to review. I was able to reproduce and have logged a bug for this. 

[Todd Shaner]

Will do–Thanks Rikk!

[johnrellis]

Todd, when you timed import with various settings, did you purge the ACR cache (Preferences > Performance Purge Cache) before each run?  I'm doing some timing and observed that standard previews build about as fast as embedded previews when you re-import the photos but go about twice as slow on an empty cache (I think I remember a comment by Chris Cox or Simon Chen a long time ago to this effect).