Keeping shadows but erasing background..

A couple of wee tips

We can strengthen tonal values in layers with things like curves and levels, and we are usually advised to use an adjustment layer.  We can't use adjustment layers for layer masks, but we can certainly apply curves and levels directly to the mask.  So with the layer mask selected (It will have a white outline) use Ctrl L (Cmd L) or Ctrl M (Cmd M) to open the levels or curves dialog window.  Moving the black slider towards the centre will firm up the blacks.

Alternatively, set the brush blend mode to Overlay.  Painting with black will only affect pixels that are currently 50% grey or darker, and painting with white will will only affect pixels lighter than 50% grey.

You can reveal the layer mask full screen by Alt (Opt) clicking it.  This is often an easier way to see any wee gaps etc.  You can work on the mask while in this mode.  Alt clicking the mask a second time will exit that view.

BTW Use whatever selection tools work best for you, and don't be afraid to mix and match on the same selection.  I now use Select Subject or the Object Selection tool as my innitial selection method in 95% of cases.  If you add a layer mask with that selection active, you can then use clipping paths, Quick Select, Quick Mask etc to clean up the mask.

If you have the time, think about joining in our SFTW threads that have just started again.  These are an excellent way to practice your composite skills, and get feed back and advise, and they are a whole lot of fun.

https://community.adobe.com/t5/photoshop/something-for-the-weekend-part-89-lockdown/m-p/11056311?page=1

https://community.adobe.com/t5/photoshop/something-for-the-weekend-part-90-shop-till-you-drop/m-p/11084005?page=3#M326041

Eyeball with a lasso around objects, Shift + ctrl + i to invert selection, press delete to clean it all up.

To verify if a Layer has »stray pixels« it canbe useful to apply the Layer Style »Stroke« set to »Outside« and then the Burn Tool set to »Shadows« often provides a quick solution. 

Did you punch out the objects in the Shadow Layer’s Layer Mask?

That would not be necessary and could cause other issues.