Update link to a specific layer in Photoshop

I have a linked Photoshop file from which I'm using individual layers in frames spread all around my document.
However, when I add another layer in the Photoshop file and relink, the warning "The file xxx.psd has different layers than the imported version. The link will be updated with the source file's current layer visibility settings" pops up and then makes all layers from the Photoshop file visible in the individual frames in the document rather than just the specific layer I initially placed. Thus forcing me to manually relink said layer from the Photoshop file to its respectable frame in the InDesign document.

Is there any way to avoid this? Attached is before and after updating the link.

I hope it all makes sense(?)