P: Duplicates detection is not working well for images with uppercase accent letters

[j.rypacek]

Hi,



I've found a bug in Lightroom Classic, including the newest version. I'm using Windows 10.

Please don't tell me that accents are special characters and software can't handle them. We are in unicode world and any char is easy to handle.


 

So what's the problem?

Duplicates detection is not working well for images with uppercase accent letters in the filename and it will import the same images multiple times. Detection is working fine with lowercase accent letters in the filename.

I've tried "synchronize folder" and if there is any new image, it will import the image and than every image with uppercase accent letter... Usual import dialog has the same problem.

 

How to reproduce?

Let's have any image in LR catalog. Rename to Á.jpg. Click on Import. Choose directory which contains this image and check "don't import duplicates". This image with uppercase accent letter will be imported anyway.

 

Which characters?

I've tried "Á, Č, Ě", but there will be much more letters.Probably from other languages too... See https://en.wikipedia.org/wiki/Czech_orthography#Alphabet

Lightroom Classic version: 7.5 [ 1186149 ]
License: Creative Cloud
Language setting: en
Operating system: Windows 10 - Business Edition
Version: 10.0.17134
Application architecture: x64
System architecture: x64
Logical processor count: 6
Processor speed: 3,6 GHz
Built-in memory: 7605,9 MB
Real memory available to Lightroom: 7605,9 MB
Real memory used by Lightroom: 954,6 MB (12,5%)
Virtual memory used by Lightroom: 974,6 MB
GDI objects count: 664
USER objects count: 2297
Process handles count: 1648
Memory cache size: 180,3MB
Internal Camera Raw revision: 1001
Maximum thread count used by Camera Raw: 4
Camera Raw SIMD optimization: SSE2,AVX,AVX2
Camera Raw virtual memory: 326MB / 3802MB (8%)
Camera Raw real memory: 327MB / 7605MB (4%)
System DPI setting: 96 DPI
Desktop composition enabled: Yes
Displays: 1) 2560x1440
Input types: Multitouch: No, Integrated touch: No, Integrated pen: No, External touch: No, External pen: No, Keyboard: No

[Rikk Flohr: Photography]

I am not able to duplicate what you are seeing in 10.13.6 with Lightroom 7.5. It immediately recognizes the second import attempt as a duplicate. 

I tested on Windows 10 as well and Lightroom functioned correctly there also. 

[j.rypacek]

Hi, thank you for your time. Let's try it again, please.
I've just reproduced this using following scenario:

1. prepare 4 images (I've used copies of the same image) with names: A.jpg, Á.jpg, c.jpg, č.jpg
2. import this images to Lightroom (I've used drag & drop)
3. open location of imported images (I've used show in explorer)
4. try to import this images again (I've used drag & drop) - duplicates detection will work and no images are being imported
5. rename A.jpg to A2.jpg, Á.jpg to Á2.jpg, c.jpg to c2.jpg, č.jpg to č2.jpg using Lightroom
6.  try to import this images again (I've used drag & drop) - duplicates detection won't work and one image (Á2.jpg) is going to be imported.

Thank you again
Josef

[Rikk Flohr: Photography]

I followed your steps Josef.

In Step 6 Lightroom recognized all 4 images as duplicates and there were no images imported.  I still cannot reproduce what you are seeing. 

[j.rypacek]

I have been able to reproduce this on 2 computers... I'm using windows 10 with Czech language. I'll try to reproduce this on another English version of windows 10...

[j.rypacek]

I tried to reproduce this on windows 10 EN. It's the same, but while recording I ends with different situation (photo is not visible twice, but there is one empty photo). When photo was visible twice - show filename was the same and when I deleted one, the other disappeared too.

Please see attached video. From 0:40 to 1:10 I'm looking for rename function. At 2:30 I restarted Lightroom.
Also at 0:33 you can see highlighted image, which will be 

[Rikk Flohr: Photography]

If you look at the file size on disk of C-3 before you change the name and after you change the name, does the file size change at all? In the icon view your video shows, I cannot see enough detail on the individual files. 

[j.rypacek]

No, filesize is the same and image file is not affected at all.
The only problem is in Lightroom - photo is not recognized as duplicate and then photo is added multiple times and catalog can be corrupted then...

[johnrellis]

Your recipe produces the same results on my LR 7.5 / Windows 10 / English computer -- the file Á2.jpg gets imported twice. Here's a screen recording:
https://www.dropbox.com/s/0em2ka8cpoz1k5s/rename-import-duplications.2018.09.13.mov?dl=0

[johnrellis]

Here's a likely explanation of the bug. To do a case-insensitive comparison of the filenames, this code path is calling the Lua function string.lower() rather than LrStringUtils.lower() -- the latter is Unicode-aware, the former is not.   string.lower ("č") produces the intended result (a lower-case string), whereas string.lower ("Á") produces an incorrect result, an upper-case string:



This could be fooling LR into thinking the file Á2.jpg is not duplicate. (I tripped over this gotcha when I first started writing plugins.)

[Rikk Flohr: Photography]

Ok, I was finally able to get it to reproduce. I've logged it for the Lightroom Classic team to fix.

[j.rypacek]

Thank you Rikk!

[FeedbackCommunityMember]

Hello,
I've got the same issue with LR 8.0 on French Windows 10.
Do you know in which release this issue will be fixed?
Thanks,
Denis

[j.rypacek]

Still not fixed in 8.1.

[Rikk Flohr: Photography]

Greetings

Updates to Lightroom Classic CC (8.2), Lightroom CC for desktop (2.2), Lightroom CC.Mobile (Android and iOS) were released on Monday, February 11, 2019. Please verify that your issue is fixed by the current update.

Thank you for your patience.

[j.rypacek]

Hi Rikk, it looks it really works!
Thank you for your cooperation

[warnerg4076398]

- Import a photo.
- Rename it so it has the letter É in it (like Église, Évènement or CAFÉ)
- On the next import, Lr will assume the photo is a new one ans suggest to import it even if the New Photos Only option is checked. If you reimport it, Lr will then crash as there are two instances of the same photo in the database.

Note that é works fine. The bug only happens with É.

[johnrellis]

This bug was reported fixed in LR 8.2.  Do the menu command Help > System Info and verify you're running 8.2, and if not, do Help > Updates.