imported video changes frame rate

The issue is I import a video at 4fps to my 24 fps project, and match the clip to the document frame rate. Works fine and the project runs at 24 fps. But when I close it and reopen the project runs at 4fps. 

When I hide the 4fps layer the project runs at 24 fps like I want it to.

Anyway to change this so the layer still runs at 24 fps?