Face recognition flagging thousands of photos that are completely unsimilar

i started a big campaign to tag al lthe 40,000 images in my catalog. So far I've tagged 2 images of my wife so you woudl think it woudl know her pretty well. Whne i go to Similar images it is showing me 16,000 faces almost none of which are my wife and including thing slike black men when she is a white woman!!!  Occassionally I'll find one. Why doesn't it put the better matches at the top. isn't there some cut off at which they are no similar at all? 

I can see any way to select all of them and say no it isnt her? Inthe similar modeule only option is either to them a differnet name.

Just seems really rubbish at finding simialr faces despite having over 1,000 named examples for each of my family.