P: AI Filter fails to find person of color in group photo

[Olivia309883589ozp]

When using the Lightroom Classic 12.4 'Subject' AI filter, it doesn't recognise the only person of colour. This is terrible, and racist. I hope by reporting this it might contribute to technical improvements, as this isn't the firrst time I've had the same issue with AI not recognising people of colour.
Very disappointing.

[Rikk Flohr: Photography]

Thank you for the report. We take these types of reports very seriously. I am forwarding to the team for review. 

To assist us, can you supply the original file used to generate this selection result? 

Do you see the same issue when using Camera Raw instead of Lightroom Classic? 

[Rikk Flohr: Photography]

Setting status to investigating

[Olivia309883589ozp]

Hi, yes same results in Camera Raw. How do I send you the original file?

[Rikk Flohr: Photography]

Share it to a service (Creative Cloud Folders, Dropbox or similar) and then Direct Message me the link. 

[John R Ellis]

I was curious how frequently that disturbing issue might occur. I googled for wedding photos with people of multiple racial backgrounds and downloaded 80. LR's People masking did very well, e.g.

 

Only one photo had errors, missing some small, partially obscured faces (four white and five black):

You can download the test set and corresponding screenshots of People masking here:

https://www.dropbox.com/s/o4arkekv5fj2uc7/multi-racial-wedding-photos.2023.07.10.zip?dl=0

Early face-recognition algorithms were well-known to have higher error rates with non-whites, but that problem was fixed a decade ago with better training sets.  I'd be very surprised if there were any gross biases in the Adobe training sets used for People masking.

[simonsaith]

Thanks for sharing your original with us. I could reproduce the select subject issue from my end. For this type of portrait/group photos, the select people tool is also available in Lr and could be a better choice because it is specifically optimized for people selection. The select people tool allows you to select individuals in a group photo and "All Persons", which seems to do a much better job (both in terms of higher mask quality and selecting all and each individuals) in this particualr example than the general purpose select subject tool. 

As Adobe, we take these kind of AI ethics issues very seriously and all AI features in Lr have considered AI ethics during R&D and there is also a formal process for extensive ethics evaluation and reviews before any AI feature is released. Having said that, the AI models are not infallible in every possible case given the diverse on-device environment that it is intended to run equally well, such as lower-end iOS and Android devices. We have strived to minimize such failure cases as much as possible given the device constraints.

[John R Ellis]

[This post contains formatting and embedded images that don't appear in email. View the post in your Web browser.]

"The select people tool ... seems to do a much better job (both in terms of higher mask quality and selecting all and each individuals) in this particular example than the general purpose select subject tool."

I used Select Subject on my test set, and in general, it didn't do nearly as well at Select People, and it made similar errors with whites, blacks, and asians. On this small test test, there is no indication the errors are biased against one or more races.

Here are some of the most blatant errors made by Select Subject (where Select People got all the photos correct).

It misses three white faces and half of a white face and half of a black face:

It misses all of a white face and part of a black one: 

It misses a black face:

It misses a Chinese face:

 

It misses two white arms and one black arm:

It misses the part of a white face and most of the body: