Grid Photo/Media Replacement

I'm clicking on one of the grid photos and I click Media to upload a new photo to replace the original. The photo will not load into the grid, instead, it is on top as a new layer. Please add option when clicking a photo in a grid to replace photo or delete photo to leave a blank space.