If you have multiple images selected and you click on the actual image what you are doing is changing the Primary selected image and not Deselecting all the others. If you multiple images selected to Deselect all the others you have to click on the boarder of the image you want to leave selected.
You can do all that in the Film Strip area in the Develop module so you don't have to go back to the Library module.
But that is what you have to do.
I suggest the only time you select multiple images if you want to apply the same develop settings to all of them.
Once you complete that select the boarder of the one you want to stay selected and be displayed in the Develop module.