I am new to photoshop and editiing, I am trying to add a PNG but the background is still there and have tried to remove it. Its the transparant one that is suppose to let me layer.
I can see a checkerboard pattern in your screenshot. The checkerboard represents transparency in Photoshop.
Could you please post a screenshot of the entire window with the Layers panel expanded and visible? Also, please describe the steps you are taking and explain the outcome compared to your expectations.
You have downloaded a vector file (AI/EPS) and opened it in Photoshop, which is raster, so everything is on one layer.
What you need to do instead is open the image in a vector application such as Illustrator. Then open the Layers panel and turn off or delete the bottom layer that consists of tiny repeating squares. After that you can open it (or copy and paste) from Illustrator to Photoshop.
The squares are simulating transparency, but they are really squares with alternating colors, as you will see when you open the file in Illustrator.