Remove Background - Refining BG Area

I'm working on removing the dark background on a photo of cardinal flowers. The button under Properties>Quick Actions gets rid of a lot of the background but leaves blobs of black all throughout the bright red flowers.

When I double click on the layer mask, I can hover and even ctrl+click to see a pink outline that is EXACTLY what I would like to keep, but I can't figure out the next step to apply that to the image and erase what's outside of it. Help!