Lightroom 3.3 Performance Feedback

[Tom Hogarty]

Please use this discussion topic for your feedback on Lightroom 3.3 RC and the final Lightroom 3.3 release when it becomes available.  The Lightroom team has tried very hard to extract useful feedback from the following discussion topic but due to the length and amount of chatter we need to start a new, more focused thread.  Please post specifics about your experience and be sure to include information about your hardware configuration.

Regards,

Tom Hogarty

Lightroom Product Manager

[Sean McCormack]

Wrong.

While the first 2 betas were on Mac, the team have PC and Mac sections

As for Adobe in general, there plenty of PC only apps.

[areohbee]

thewhitedog wrote:

... I never got a satisfactory answer to my question about the relationship between the Lightroom 3 Catalog.lrcat file and the Lightroom 3 Catalog Previews.lrdata file. Perhaps no one knows, but I'm curious; knowing more about these files might help with troubleshooting Lightroom.

Lots of people know... - you may not be wet behind the ears mac-tech-wise, but you've just revealed some moistness Lightroom-wise.

lrcat is a sqlite database file - it contains all the develop and metadata settings for your photos, as well as collections, pick flags, stacking... - all in relational database tables.

lrdata is a folder of jpeg previews, together with the bookeeping necessary to use them - I don't know too much of the details...

Rob

[thewhitedog]

@ Rob -

Thanks for the information. Though I don't know how many people "lots" is - I had to ask the question twice before I got an answer. As for Lightroom, I've been suing it since the first beta came out. As is clear on this forum, there is plenty we all don't know about the program. That quip about moisture behind my ears, though probably intended humorously, was both ungracious and unnecessary.

[areohbee]

thewhitedog wrote:

That quip about moisture behind my ears, though probably intended humorously, was both ungracious and unnecessary.

Sorry to have offended.

[JayS_In_CT]

thewhitedog wrote:

<snip>

Now that I've spoken up again, I expect more criticism. But until someone from Adobe rules out hardware as a source of difficulty, I, for one, will remain unconvinced. The real culprit may be the manufacturers who have often failed to keep drivers for their products up-to-date. This became such a problem on the Mac in recent years that Apple has taken over most of the driver support for the cards designed for their computers. Hardware companies are all too often poor software designers. This is hardly news, and the issue goes far beyond graphics card support.

I hope Adobe doesn't rely solely on users to determine which graphics cards work and which do not. Adobe is in a far better position than their customers are to test the various possible graphics card configurations.

<snip>

I agree with your post whitedog... both on the driver front (we've seen folks who have upgraded drivers have issues go away) and just on the sheer performance of a hardware component.  Add to that, the complete variability of how different hardware components interact with a system as a whole.  Keep in mind, unless you build your own system, you are totally at the whim of the manufacturer to purchase motherboards, graphics cards, drives, CPU, power supplies.. the list is endless and the cheaper they can get it, the faster it will go into production.  Add to this further, is the interactions of all the apps an individual may have running, conflicts for resources there, etc.

On the Apple side though, things go "out of style" a bit too quick in my opinion.  I really enjoy my Macbook Pro, but I can't remember the last time mine had a decent hardware upgrade.. and I can't rememeber seeing any driver updates to the X1600 ATI graphics in mine.. Perhaps buried in some OS update, but when I looked for any info on google...there seemed to be none.

So the problem for Adobe on this, is how to test for every possible permutation.  They obviously can't.. which in turn leads to some of the "it's failing on my machine, but not on yours" issues. This has been a software developer's nightmare forever, and I don't think it's going away anytime soon.

Jay S.

[George in Seattle]

I just installed 3.3 and I notice it handles spot movement very differently from 3.2, and in a less desirable way. --it no longer updates on the fly as you drag!  Or as you resize.  It waits until you release the mouse button, which makes it very difficult to align the seams of a correction.  That change looks like an attempt to make it faster but at the expense of functionality.  --Except it is not any faster; the entire 3.2 wait time appears to be shifted to the point after which you release the mouse button.   The only thing gained in 3.3 is that a drag no longer locks up in mid-drag, but this is at great expense, IMO.

[Lee_Jay-ZyZk56]

George in Seattle wrote:

I just installed 3.3 and I notice it handles spot movement very differently from 3.2, and in a less desirable way. --it no longer updates on the fly as you drag!

It does for me.  Just tried it.

[George in Seattle]

You are correct --my mistake.  It does update in mid-drag, but in much slower steps.  I was not patient enough to notice that.

[Ulr]

Your last post covers another area - spot and healing - and I don't see any lag while updating the spot circles. And this is on an average office desktop. Did you try with lens corrections turned off ?

Rgds, Gilles.

[George in Seattle]

THANKS! Turning off the lens profile makes a HUGE positive difference. Both in speed and less weird jerky movements. That makes this tool highly usable again  

[Ulr]

George, can you confirm that you're using Lr 3.3 ?

Because this version is supposed to have a fix for the spot healing performance bug.

The trick to deactivate lens correction was used for previous versions.

Rgds, Gilles.

[Ian Lyons]

Ulr wrote:

George, can you confirm that you're using Lr 3.3 ?

Because this version is supposed to have a fix for the spot healing performance bug.

The trick to deactivate lens correction was used for previous versions.

Any advice given re turning lens corrections off to improve the responsiveness of spot heal and adjustment brush was not related to any bug. Unfortunately, lens corrections and the auto mask feature in adjustment brush place a very significant burden on the CPU, which is why it is best to apply the former late in the editing process and the latter only when absolutely necessary.

[George in Seattle]

Yes it is 3.3.  I put it on 2 days ago and can confirm it says "3.3" in "About".   Regarding Auto mask: Doesn't Auto mask tax the CPU only when you are actually painting?

[thewhitedog]

It does appear we are getting closer to some real solutions. I don't use Lens Correction - my Canon lens profile doesn't change anything so there's not need for it. I do use the auto mask feature occasionally, but not often. And I'm not using a Windows PC so I don't have any registry issues. Color me lucky. At any rate, this begins to explain why I'm having few problems and why others are having drastic ones.

[Samoreen]

Hi,

George in Seattle wrote:

It waits until you release the mouse button

Not exactly the same for me here. No target update occurs until I stop moving the spot source for a while (clone mode or heal mode), not until I actually release the mouse button. Probably another symptom of the performance problem discussed elsewhere.

[areohbee]

George in Seattle wrote:

Anyone can duplicate this problem by slapping about that many dust spots against any uncropped raw image.

Don't be so sure - I can put a dozen dust spots on an image with no noticeable slowdown whatsoever.

Consider posting a link to the raw+xmp file, so others could check it...

[drichi09]

After being very hesitant to go to 3.3 from 3.2, which worked reasonably well (though much slower than LR 2 series) except for the problem with watermarks that would suddenly stop working for some fonts, I changed when the RC was removed.

With no changes to my system except for the change to 3.3 I have began having problems with speed and with hanging. Generally, this is in the developement mode where I notice it most. Today, while checking presets on some photos it began hanging when I did nothing execpt run my mouse over the presets to look at the small preview screen. It would run my processor up to 100% just the show the miniscule preview! Usually I have to actually apply the preview to really tell anything, but it bogged down to the extent it took 10 or more seconds to apply or remove the preset from the file.

I optimized the catolog, which has always been akin to rebuilding permissions on a Mac---something to do to keep one from throwing the machine out the window until you calm down, but fixing nothing---with no change. I restarted LR a number of times. No change. Deleted cache. Naturally, no change. Rebuilt premissions and restarted just for grins and giggles. No change.

The improved processing, sharpening, and noise reduction are nice, but frankly the continued problems with LR 3 make it not worth the trouble, especially when I can get equal image quality and fewer headaches with other RAW processors.

iMac 10.5.8, 8gb etc. Should make no difference since it worked with 3.2....

[areohbee]

drichi09 wrote:

...especially when I can get equal image quality and fewer headaches with other RAW processors.

Which ones?

[JW Stephenson]

Tom/George/Agnus1 & Others,

Warning: This is a long post.

I may have found a workaround for the problem I have been having with system lags and hangs, particularly during the use of the spot healing tool.  It is a little early to tell but I am optimistic.

After watching the CPU meter during periods where LR would hang after adding a new spot or modifying the size of the existing spot from the spot healing tool, I kept noticing that during the hang time one of the 16 cores would be at 100% and the other were fairly much not operating at all.  When the hang would break, all the CPUs would then become partially and randomly utilized.  Very strange.

After some internet searching I found out about a program called the Windows Event Viewer (type Event Viewer in the start menu).  From this lead, I found the Windows Event Viewer was recording massive"information" entries with "amdkmdag" listed as the source,"62464" as the Event ID and "DVD_OV" as the Task Category.  The number of entries was easily hundreds a day and many times 20-100 in a single second, and the second I was looking at was when LR was hung up with the spot removal tool.

A little more research of this "amdkmdag" service lead me to finding it was called the "ATI External Event Utility" and given its name, probably related to the ATI graphics card I have installed.

After digging deeper into what overhead runs in background on account of the graphics card software and drivers, I decided on a hunch to disable all that I felt was of little current value, but especially the ATI External Event Utility.  So I did two things:

1.   Disable ATI External Event Utility.  This was accomplished in the registry as follows:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Atierecord]

"eRecordEnable"=dword:00000000

I learned how to do this from a discussion at:  http://209.147.121.98/board/showthread.php?t=33960196&mode=linear

2.   Remove unimportant (to me) graphics card services (5 total) from start-up:

CCC.Exe,Atiedxx.exe, Atiesrxx.exe, Mom.exe and LIStart.exe

I learned how to do this from a discussion at:  http://www.ghacks.net/2010/01/07/what-are-ccc-exe-atiedxx-exe-atiesrxx-exe-mom-exe-and-clistart-exe/

The result is that my performance in the Develop module is much more as I expected when upgrading (slightly slower than 2.5 but still pretty zippy given the horsepower in my system).  In particular, I am now rarely getting a hang up with the spot healing tool and if I do it is only for a couple of seconds.  Back in business!

I probably didn't need to do both steps as they both probably disable the event messaging but I don't want to figure that out right now since my system is again functional and I really don't want to go backwards at this point.

This might help some of you out there with ATI cards running Windows 7.  I will be real disappointed if my problem returns down the road, but not completely surprised.  If George and Agnus1 could give this a try that might confirm if the workaround is useful for systems with similar hardware.  This may only be a problem with W7/ATI systems or it may exist in other combinations but less noticeable.

As a closing note, I have had to disable an administrative service utility as a workaround to my system lag problem.  That does not mean it solved the problem itself and I will of course share this with my bug reporting contact at Adobe as well so they can look into the functionality of this service utility relative to version 2.5 where it didn't appear to be causing any issues at all.

Best of luck to all.

Jeff

[thewhitedog]

@ JW: Wow! This has driver compatibility problems written all over it. Which means it's not, strictly speaking, a hardware issue but a software problem with ATI drivers. The question then becomes, how do we get ATI to keep their drivers up-to-date? We cannot simply assume that if we buy a new graphics card that the drivers will be any better. It will probably take Adobe and Microsoft together leaning on ATI to get their head back in the game. Another question comes to mind: Are NVidia cards (and software) any better?

[areohbee]

thewhitedog wrote:

...which means it's not, strictly speaking, a hardware issue...

When I used to work on electrical power generation and distribution control systems, I worked on the firmware in the remote control units. The people who worked on the master station software referred to the entire remote control unit as "hardware", even though it was mostly software.

Moral of story: maybe best not to get too "strict" about terminology...

;-}

PS - You've put the blame on ATI's driver, but even though the problem surfaced in the driver, it still could be a Lightroom problem at its roots - technically speaking...

R

[ChBr02]

Perhaps, I can add some credence to the NVIDIA v ATI scenario.

I have two systems that I run Lr on:

1)  A Dell XPS 710 with dual core 4GB using XP Media Center SP3.  It has an NVIDIA GeForce 8600 GTS graphics Card w/256 MB memory attached to two 1680x1050 monitors.  I have been using Lr on this system since Lr1.0.  It wasn’t until 3.0 that I had any performance issues.  3.0 on this system was basically unusable for me. Slider operations were so slow that a minute movement could take as long as 1-3 seconds to register.  3.2Rc, 3.2, and 3.3 were each progressively better.  There are other performance issues, but the sliders now work almost as quickly as 2.7.

2)  A Dell Studio 15 with I7 quad hyperthreading/Win 7 Ultimate/6GB/500GB 7200, etc.  It has:

ATI Mobility Radeon HD 5470

      Memory      1024 MB

      Memory type 2

      Driver version    8.680.0.0

I started Lr on this system at Lr 3.0 hoping it would be faster than the other system.  No such luck.  About the same.

On “both” systems with 3.0 through 3.2, trying to clone using the spot tool was very precarious.  If I moved the spot tool as little as 1/3 of the height or width of the photo, the system would lockup requiring a reboot.

Reading the scenarios between NVIDIA and ATI on here, I started trying to break both systems with outrageous demands on the spot tool.  I tried various sizes reaching the span of the photo, and changing sizes, etc. with a gazillion other edits.  Here are the results:

On system 1) with 3.3 (NVIDIA):

While all spot tool operations on 3.3 were slightly slower than on 2.7, it is basically something I can live with.  After about 50 spot operations, I started to get 2-3 second lockups.  On 2.7, I used to use 100+ spots to restore antique photos with no noticeable slowdowns.  But, even after all of these edits, the sliders are still fast enough so that I won’t be going back to 2.7.

On system 2) with 3.3 (ATI):

It only took about 20 or so operations before I started having problems.  At first I thought the lockups would require a reboot.  But, then, I found out that the lockups weren’t permanent, and that I regained “control” in about 14-15 seconds.  But, that is still unacceptable.  It did not seem to affect the speed of other edits, however.  Which once again aren’t as fast as 2.7 on system 1), but livable.

On both systems, rendering, imports, and exports can be slow enough to be troubling.  But, the occurrences are so variable, that I can’t pin down a particular problem area.  I just expect that I will have to wait, especially on exports, so I do something else in the meantime.  It is a little difficult to live with unbearably slow sliders.  Thankfully, those are better.

But then, there are a lot more differences between these two systems than just the brand of their graphics cards.  The combitorial effects of all the hardware, all the software, the drivers, and the sequence in which any of this loaded is way up in the trillions.

Charlie

[Agnus1]

Very interesting Jeff. Unfortunately at the moment I just don't have time to go altering the registry of my computer. Fortunately I don't have any clients waiting for orders and I won't have until after the first of the year. I have other irons in the fire that are keeping me too busy to be jumping into my system registry. I am always a bit wary to do registry altering, although if I need to I will. When I get some time I'll let you know how it goes and if I see any improvement in Lightroom performance. I really appreciate you digging this information up. This fix sounds much more appealing than reinstalling windows! I really dislike when that solution if offered to a program performance problem. In 30 years of owning a computer, I've had to reinstall a few opperating systems but it isn't something I do lightly. It is a time consuming pita! Again, thank you Jeff for the information.

[Samoreen]

Hi,

JW Stephenson wrote:

After some internet searching I found out about a program called the Windows Event Viewer (type Event Viewer in the start menu).  From this lead, I found the Windows Event Viewer was recording massive"information" entries...

This post gave me the idea to monitor what lightroom.exe is doing with the system when editing. I used the well known sysinternals tools for Windows (File Monitor and Registry Monitor). I disabled all plugins. I noticed the following:

Even when doing nothing (LR window minimized), lightroom.exe accesses the registry every 10 seconds and reads the following keys:

QueryKey    HKCU\Software\Classes    SUCCESS    Name: \REGISTRY\USER\S-1-5-21-1875880806-976815447-3821793529-1143_CLASSES   
OpenKey    HKCU\Software\Classes\Drive\shellex\FolderExtensions    NOT FOUND       
OpenKey    HKCR\Drive\shellex\FolderExtensions    SUCCESS    Access: 0x8    
QueryKey    HKCR\Drive\shellex\FolderExtensions    SUCCESS    Name: \REGISTRY\MACHINE\SOFTWARE\Classes\Drive\shellex\FolderExtensions   
08148193    OpenKey    HKCU\Software\Classes\Drive\shellex\FolderExtensions    NOT FOUND       
EnumerateKey    HKCR\Drive\shellex\FolderExtensions    SUCCESS    Name: {fbeb8a05-beee-4442-804e-409d6c4515e9}   
QueryKey    HKCU\Software\Classes    SUCCESS    Name: \REGISTRY\USER\S-1-5-21-1875880806-976815447-3821793529-1143_CLASSES   
OpenKey    HKCU\Software\Classes\Drive\shellex\FolderExtensions\{fbeb8a05-beee-4442-804e-409d6c4515e9}    NOT FOUND       
OpenKey    HKCR\Drive\shellex\FolderExtensions\{fbeb8a05-beee-4442-804e-409d6c4515e9}    SUCCESS    Access: 0x1    
QueryKey    HKCR\Drive\shellex\FolderExtensions\{fbeb8a05-beee-4442-804e-409d6c4515e9}    SUCCESS    Name: \REGISTRY\MACHINE\SOFTWARE\Classes\Drive\shellex\FolderExtensions\{fbeb8a05-beee-4442-804e-409d6c4515e9}   
OpenKey    HKCU\Software\Classes\Drive\shellex\FolderExtensions\{fbeb8a05-beee-4442-804e-409d6c4515e9}    NOT FOUND       
QueryValue    HKCR\Drive\shellex\FolderExtensions\{fbeb8a05-beee-4442-804e-409d6c4515e9}\DriveMask    SUCCESS    0x20   
CloseKey    HKCR\Drive\shellex\FolderExtensions\{fbeb8a05-beee-4442-804e-409d6c4515e9}    SUCCESS       
EnumerateKey    HKCR\Drive\shellex\FolderExtensions    NO MORE ENTRIES       
CloseKey    HKCR\Drive\shellex\FolderExtensions    SUCCESS       
OpenKey    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\CPC\Volume    SUCCESS    Access: 0x2000000    
OpenKey    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\CPC\Volume\{2c18e6e0-787e-11db-ad7d-0015f2553462}\    SUCCESS    Access: 0x2000000    
CloseKey    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\CPC\Volume    SUCCESS       
QueryValue    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\CPC\Volume\{2c18e6e0-787e-11db-ad7d-0015f2553462}\Generation    SUCCESS    0x1   
CloseKey    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\CPC\Volume\{2c18e6e0-787e-11db-ad7d-0015f2553462}\    SUCCESS       
OpenKey    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\{2c18e6e0-787e-11db-ad7d-0015f2553462}\    SUCCESS    Access: 0x2000000    
QueryValue    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\{2c18e6e0-787e-11db-ad7d-0015f2553462}\_LabelFromReg    SUCCESS    "NTFS-2"   
CloseKey    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\{2c18e6e0-787e-11db-ad7d-0015f2553462}\    SUCCESS       
OpenKey    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\CPC\Volume    SUCCESS    Access: 0x2000000    
OpenKey    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\CPC\Volume\{2c18e6e0-787e-11db-ad7d-0015f2553462}\    SUCCESS    Access: 0x2000000    
CloseKey    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\CPC\Volume    SUCCESS       
QueryValue    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\CPC\Volume\{2c18e6e0-787e-11db-ad7d-0015f2553462}\Generation    SUCCESS    0x1   
CloseKey    HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MountPoints2\CPC\Volume\{2c18e6e0-787e-11db-ad7d-0015f2553462}\    SUCCESS   

Since this information is static, I wonder why it must read it again every 10 seconds.

Even more astonishing

As soon as I start using the local adjustment brush, lightroom.exe is continuously accessing the following keys

OpenKey    HKCU    SUCCESS    Access: 0xF003F    
CreateKey    HKCU\Software\Microsoft\Windows NT\CurrentVersion\Time Zones    SUCCESS    Access: 0xF003F    
CloseKey    HKCU    SUCCESS       
QueryValue    HKCU\Software\Microsoft\Windows NT\CurrentVersion\Time Zones\TimeZoneKeyName    SUCCESS    "Romance Standard Time"   
OpenKey    HKLM\Software\Microsoft\Windows NT\currentVersion\Time Zones\Romance Standard Time    SUCCESS    Access: 0x20019    
QueryValue    HKLM\Software\Microsoft\Windows NT\currentVersion\Time Zones\Romance Standard Time\Std    SUCCESS    "Romance Standard Time"   
QueryValue    HKLM\Software\Microsoft\Windows NT\currentVersion\Time Zones\Romance Standard Time\Dlt    SUCCESS    "Romance Daylight Time"   
QueryValue    HKLM\Software\Microsoft\Windows NT\currentVersion\Time Zones\Romance Standard Time\Tzi    SUCCESS    C4 FF FF FF 00 00 00 00 ...   
CloseKey    HKLM\Software\Microsoft\Windows NT\currentVersion\Time Zones\Romance Standard Time    SUCCESS       
CloseKey    HKCU\Software\Microsoft\Windows NT\CurrentVersion\Time Zones    SUCCESS       
OpenKey    HKLM\Software\Microsoft\Windows NT\currentVersion\Time Zones\Romance Standard Time    SUCCESS    Access: 0x20019    
QueryValue    HKLM\Software\Microsoft\Windows NT\currentVersion\Time Zones\Romance Standard Time\Std    SUCCESS    "Romance Standard Time"   
QueryValue    HKLM\Software\Microsoft\Windows NT\currentVersion\Time Zones\Romance Standard Time\Dlt    SUCCESS    "Romance Daylight Time"   
QueryValue    HKLM\Software\Microsoft\Windows NT\currentVersion\Time Zones\Romance Standard Time\Tzi    SUCCESS    C4 FF FF FF 00 00 00 00 ...   
CloseKey    HKLM\Software\Microsoft\Windows NT\currentVersion\Time Zones\Romance Standard Time    SUCCESS       
OpenKey    HKLM\Software\Microsoft\Windows NT\CurrentVersion\Time Zones\Romance Standard Time\Dynamic DST    NOT FOUND

When I say continuously, this really does mean it doesn't stop accessing the same keys. As soon as it is done with the last line of the above block, it starts again. This madness stops as soon as I stop using the local adjustment brush. Now tell me what this tool has to do with the Time Zone settings of my system. Here again, this is static information, so there's no need to read it again and again. It won't change during a Windows session.

Performance problems you said? No wonder that LR becomes slow if it spends time doing useless things with the registry.

[Samoreen]

Hi,

Samoreen wrote:

Even more astonishing

As soon as I start using the local adjustment brush, lightroom.exe is continuously accessing the following registry keys

More about this...

The above mentioned registry keys are read each time I'm moving the local adjustement brush cursor. If I stop moving the mouse, LR stops reading these keys. As soon as I start creating new brush strokes, the registry keys are read again.

EDIT: It seems that the problem is not only related to the local adjustment brush. This happens systematically when I hover the image, even when not in local adjustment mode. Please consider that this represents thousands of registry reads within a few seconds.

I also noticed that this also happens when LR doesn't have the focus. If I put LR in the background and Registry Monitor in the foreground, hovering the edited image in LR still generates the registry reads.