P: Does not export all selected images

Thank for this great detective work John. Truly shows Adobe broke Lightroom 6 or CC. Kind of like when 5 came out and don't remember how long it took to export with all the actual developed images. Can only hope next update. Are you listening Adobe?

I had to start using WD backup running a continuous backup of all image drives because Acronis was error prone. This explains why my exports are occasionally error filled. For now I track all files that have an issue and Export them again appears to fix just more unpaid work.

Thank you John. You wrote "But a file viewer like Bridge may have just opened the file right after step 3 to construct a thumbnail, and if it still has the file open at step 4, LR's reopen for read/write access will fail". I use AcDsee to view the exported files, and it builds thumbnails on the run as the files are created by Lightroom. AcDsee also sees the swp files (which is shortly after deleted Again by Lightroom). So this effect could be similar to that caused by Bridge.

What an excellent piece of research John. Thanks for taking the time to do it, and to explain it so clearly.

(Now I understand why I sometimes saw an icon for an exported file appear in the target directory then almost instantly disappear.)

Adobe says they try to read everything here, though they reply much less frequently. In addition, there are moderators who try to bubble up issues to Adobe as well.

Thank you John for this research! I hope Adobe sees it. I wonder if they're seeing this thread at all since they haven't responded yet.

I traced the filesystem calls made by LR 6.0.1, LR 5.7.1, and Bridge during an export on Windows. The trace shows that indeed LR 6 has introduced a bug that will cause file viewers like Bridge and some backup programs to interfere with the export in unpredictable ways.

The trace also shows that LR 6 is writing each exported file twice, causing more than twice as much disk I/O as LR 5. This might account for reports of significantly slower exports. For example, Denni Russell reported that exports were taking 2x longer in LR 6 compared to LR 5: http://feedback.photoshop.com/photosh...

Details

I used Process Monitor to trace the file calls made by LR 6.0.1, LR 5.7.1, and Bridge CC during an export, on Windows 8.1.

Here is what LR 5.7.1 does when exporting "file.jpg":

1. Opens file.jpg for writing
2. Writes file.jpg
3. Closes file.jpg

But LR 6.0.1 adds more steps -- after writing "file.jpg", it reads it back into memory, then writes it out to a temp file "file.jpg.swp", and then renames "file.jpg.swp" back to "file.jpg":

1. Opens file.jpg for writing
2. Writes file.jpg
3. Closes file.jpg
4. Opens file.jpg for read/write access, read share mode
5. Reads file.jpg
6. Closes file.jpg
7. Opens file.jpg.swp for read/write access, read share mode
8. Writes file.jpg.swp (often with a slightly different number of bytes)
9. Close file.jpg.swp
10. Renames file.jpg.swp to file.jpg

There are two problems with this:

Problem 1: LR 6 is writing twice as many bytes to disk as necessary, slowing down exports. I have no facts as to why it is doing this, but as a former developer, I can guess -- an inexperienced programmer is attempting to fix some prior bug and chose an architecturally inept way of doing it. Perhaps the extra writes in steps 7-9 are rewriting the metadata as described in this bug report: http://feedback.photoshop.com/photosh.... The author of Exiftool observed that it appears that the metadata is first written in a more typical fashion, then rewritten in a more atypical fashion.

Problem 2: In step 4, LR 6 tries to reopen the just-written file with read/write access but then proceeds to just read the file. But a file viewer like Bridge may have just opened the file right after step 3 to construct a thumbnail, and if it still has the file open at step 4, LR's reopen for read/write access will fail. When it does, LR thinks some kind of file problem has occurred, deletes "file.jpg", and reports an error to the user.

Bridge is particularly aggressive at monitoring folders for new files so it can create thumbnails. The trace shows it continually trying to open newly created files, every couple of milliseconds.. So it's not surprising that when Bridge is open on the export folder, this problem happens fairly often.

But this problem is not specific to Bridge -- any file viewer, including Windows Explorer, could reasonably try to read the exported file to create a thumbnail, right after step 3. It could also happen with file utilities that are copying files in background (e.g. some kinds of backup programs).

The Fix

A LR architect should review why the file is being written twice instead of just once. Eliminating the second write would avoid the problem and speed up exports. But if it really is necessary to write the file twice, then there are two possible fixes:

a. Keep the file open from step 1 all the way through step 9, and get rid of the gratuitous rename. That way, no other program can access it until it is fully written and closed.

b. Open the file in step 4 for read access/read share mode instead of read/write access. That way, the open won't fail if another program also has the file open for reading. This fix is suboptimal, because the other program will read bytes from the file that aren't the "final" bytes (which won't be written until step 8).

I also get the "Some export operations were not performed" during export and a list. By running the selected files second time it works. I get 2-3 files wrong in 50-100 pics. I did not notice that in v. 5.7

I use Lightroom 6 stand alone

Hmmm, perhaps! I shall test this out. Thanks for the suggestion.

I don't suppose you have a Windows Explorer window opening showing the export folder? It's a long shot but there was a bug a long time back, which I believe was caused by Windows trying to build thumbnails at the same time.

Support,
I also have the problem described, and see the randomness described.
I do not have Bridge installed, I do not have Faststone installed.
It is a persistent and constant problem, which means that I have to double and triple-check EVERY export, from very small images (600px wide), to large (4000px wide).

Windows 7, LR CC.