When you want to delete something in your project, either click the item on the project canvas or select the layer in the layer stack. Then use your delete key to remove it.
The grey trashcan icon that you see when in a project is for deleting the whole page, not individual items.