How can I convert a photoshop file(with layers) to illustrator

How you would approach this depends on content but as others have written - a layered file can't be opened in illustrator. Why not save a copy and flatten it then place in illustrator. That way you save an archive copy of the Photoshop file with the layers intact (and that's a good idea given that layers increase editability)

What is the content of the Photoshop file? Why do you want to open it in Illustrator?

Does the Photoshop file have shape layers? If so they can be saved as svg and opened in Illustrator.

Does the Photoshop file have paths? If so they can be exported for Illustrator or simply copied and pasted from Photoshop to Illustrator.

Does the Photoshop file have text? Is so you can copy and paste the text into Illustrator.

Photoshop files that contain images can be placed into Illustrator.

SInce Illustrator is a Vector imaging app and Photoshop being Raster,  trying to transfer a Photoshop file into Illustrator is possible, but its not going to really be pretty.  Illustrator files can transfer to Photoshop easier than the other way around. The layers are going to be wonky and flattened.  You might be able to work some parts of it into vector shapes by doing image trace, but there's going to potentially be some little artifacts and such that are going to mess with it.  Its going to take a little work to get it to transfer over.  The simpler the image the better.  If its not using a lot of layers and its flat, simple colors with not overly complicated shapes,  you're going to have better luck. 

How can I convert a photoshop file(with layers) to illustrator

i have tried just opening the Ai file in Photoshop

So what is it – get a layered psd file into Illustrator or the other way round?

If a Photoshop file contains meaningful vector data (Shape Layers effectively) exporting an svg file should produce data that Illustrator can utilise as such.