Lightroom keywords - Smart Collections

I have an image that appears in one of my smart collections and shouldn't. The only smart collection criteria is keyword = family. The image has the keywords friend and people. I have tried everything I can think of to fix the issue. I need some suggestions. Thanks